-- Logs begin at Thu 2022-05-19 18:36:30 UTC, end at Thu 2022-05-19 20:40:40 UTC. -- May 19 19:40:57.407657 dv-cinder-840678-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 19:40:57.415701 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 19:40:57 2022] *** May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: nodename: dv-cinder-840678-11-iscsi May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: machine: x86_64 May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: clock source: unix May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: pcre jit disabled May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: detected number of CPU cores: 4 May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: current working directory: / May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: detected binary path: /usr/bin/uwsgi-core May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: your processes number limit is 47680 May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: your memory page size is 4096 bytes May 19 19:40:57.418421 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: detected max file descriptor number: 2048 May 19 19:40:57.419289 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: lock engine: pthread robust mutexes May 19 19:40:57.419289 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: thunder lock: enabled May 19 19:40:57.419289 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 19:40:57.419289 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 19:40:57.473671 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: Python main interpreter initialized at 0x558b9dc5a9c0 May 19 19:40:57.473671 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: python threads support enabled May 19 19:40:57.473671 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: your server socket listen backlog is limited to 100 connections May 19 19:40:57.473671 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: your mercy for graceful operations on workers is 90 seconds May 19 19:40:57.474046 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: mapped 403077 bytes (393 KB) for 2 cores May 19 19:40:57.474131 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: *** Operational MODE: preforking *** May 19 19:40:57.474455 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: *** uWSGI is running in multiple interpreter mode *** May 19 19:40:57.474455 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: spawned uWSGI master process (pid: 86696) May 19 19:40:57.474659 dv-cinder-840678-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 19:40:57.476131 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: spawned uWSGI worker 1 (pid: 86703, cores: 1) May 19 19:40:57.476131 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: spawned uWSGI worker 2 (pid: 86704, cores: 1) May 19 19:40:57.476131 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 19:41:03.315966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:41:03.315966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: from cryptography.utils import int_from_bytes May 19 19:41:03.316760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:41:03.316760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: from cryptography.utils import int_from_bytes May 19 19:41:03.348932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:41:03.348932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: from cryptography.utils import int_from_bytes May 19 19:41:03.349536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:41:03.349536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: from cryptography.utils import int_from_bytes May 19 19:41:03.368043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:41:03.368231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:41:03.368424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] command line args: [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:41:03.368704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:41:03.368883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ================================================================================ {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:41:03.369139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] admin_token = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.369386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] config_dir = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.369573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.369766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] config_source = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.369944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] debug = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.370154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.370659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.370750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.370953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.371200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.371432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.371612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_config_append = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.371958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.372151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_dir = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.372307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_file = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.372646 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.372914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.373169 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.373673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.373853 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.374031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.374277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.374529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.374768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.375001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.375234 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.375467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.375716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.375961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.376249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.376779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.376964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] publish_errors = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.377199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.377430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.377609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.377857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.378094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.378329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] standard_threads = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.378560 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.378806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.378982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.379157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] use_journal = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.379334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] use_json = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.379508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] use_stderr = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.379683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] use_syslog = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.379915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.380175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.380434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.380702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.380946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.381200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.381442 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.381693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.381938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.382185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.382435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.382706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.382939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.383190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.383430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.383680 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.383931 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.384171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.384451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.384715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.384960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.385197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.385453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.385726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.385981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.386232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.386465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.386698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.386930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.387168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.388020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.388275 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.388534 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.388771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.389022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.389273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.389513 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.389764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.389998 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.390236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.390473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.390704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.390936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.391167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.391400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.391637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.391888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.392127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.392360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.392621 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.392862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.393095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.393345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.393575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.393813 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.394048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.394283 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.394520 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.394757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.394986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.395241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.395476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.395715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.395947 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.396217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.396477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.396711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.396947 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.397183 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.397452 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.397690 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.397938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.398183 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.398419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.398653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.398888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.399138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.399375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.399613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.399855 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.400092 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.400327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.400585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.400822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.401057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.401319 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.401558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.401800 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.402040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.402282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.402523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.402760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.403709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.403976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.404222 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.404484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.404731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.404968 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.405216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.405454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.405708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.405902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:41:03.405972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.406102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:41:03.406293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] command line args: [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:41:03.406365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.406569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:41:03.406636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.406743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ================================================================================ {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:41:03.406809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.407005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] admin_token = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.407080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.external = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.407247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] config_dir = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.407320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.407430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.407557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.407632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] config_source = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.407769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] debug = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.407835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.408052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.408174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.password = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.408242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.408306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] auth.token = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.408369 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.408635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.408720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.408806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.408866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.408977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.409083 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.409211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.409278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_config_append = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.409420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.409486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.409593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_dir = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.409700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.409766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_file = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.409931 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.410009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.410169 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.410236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.410418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.410485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.410670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.410745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.410843 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.410909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.411016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.411146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.411256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.411391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.411497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.411629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.411728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.411899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.411996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.412157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.412231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.412384 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.412532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.412655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.412780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.412899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.413001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.413176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.413285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.413439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.413683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.413784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.413886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] publish_errors = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.413978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.414093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.414182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.414302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.414386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.414502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.414615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.414690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.414856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.414945 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.415087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.415165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] standard_threads = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.415334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.415409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.415572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.415660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.415732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.415852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.415936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] use_journal = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.416050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.416137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] use_json = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.416230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] use_stderr = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.416306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.416424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] use_syslog = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.416553 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.416655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:41:03.416841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.416936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417397 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.417882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418118 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.418869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419163 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.419955 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420277 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.420970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.421955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422186 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422423 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422757 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.422975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.423894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424305 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424640 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.424862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.425963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426656 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426885 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.426963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427125 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427364 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.427891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428349 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428522 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428610 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.428986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429215 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.429993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430853 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.430933 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431401 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431546 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.431859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432583 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.432972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.433994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.434990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435350 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435820 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.435905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436364 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.436854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.437954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.438967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.439883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440351 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] role.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] role.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.440984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.441977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.external = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442470 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.442935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443080 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.password = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] auth.token = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.443921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444038 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444538 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444619 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.444867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445089 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445738 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.445985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446223 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446774 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.446938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447670 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.447915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448655 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.448969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449128 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449369 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.449681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.450901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451538 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.451873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452262 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.452850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453006 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453470 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.connection = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.453942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454407 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454524 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454656 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454765 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.454999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455473 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.455965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456797 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.456967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.457974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.458126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.458205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.458296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:41:03.458372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.458586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.458818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.459062 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.459289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.459542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.459789 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.460021 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.460264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.460539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.460768 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.461003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.461229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.461464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.461697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.461939 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.462175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.462406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.462652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.463002 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.463125 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.463353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.463634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.463931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.464064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.464292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.464551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.464780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.465048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.465266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.466440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.466693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.467032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.467194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.467427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.467655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.467888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.468120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.468351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.468602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.468847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.469089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.469332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.469574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.469815 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.470040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.470283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.470514 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.470746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.470979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.471210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.471452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.471698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.471930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.472180 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.472422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.472666 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.472911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.473149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.473379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.473862 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.474091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.474326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.474563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.474810 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.475057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.475291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.475535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.475781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.476012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.476255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.476529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.476766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.476996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] role.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.477245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] role.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.477472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.477715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.477959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.478205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.478450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.478692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.478934 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.479177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.479420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.479649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.479892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.480135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.480378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.480648 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.481176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.481415 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.481696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.481942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.482186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.482430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.482662 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.482914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.483162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.483399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.483624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.483852 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.484079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.484329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.484602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.484845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.485090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.485323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.485549 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.485780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.486014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.486259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.486498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.486744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.486990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.487231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.487473 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.487712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.487956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.488188 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.488449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.488680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.488922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.489164 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.489388 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.489625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.489915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.490016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.490150 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.490246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.connection = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.490370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.490468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.490625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.490857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.491093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.491401 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.491636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.491867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.492099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.492340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.492599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.492837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.493069 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.493313 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.493548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.493784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.494010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.494256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.494488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:41:03.494654 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86704) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:41:03.499997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.521692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.521925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.522123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.529634 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.529863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.532211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.550535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.550758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.584547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.589891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.636684 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.636919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.638447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.638675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.705776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.706002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.706229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.706447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.706664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.706878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.708050 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.708277 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.708528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.708743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.708954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.709172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.732958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.733277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.733418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.733626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.733841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.733924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.734729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.735047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.735360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.745801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.746066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.758879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.759102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.759323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.759542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.759751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.759956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.760162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86704) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.760673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.760908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.761133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.761352 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.761568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.761782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.761995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:41:03.777428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.777706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:41:03.778713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.778985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.778985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:41:03.779286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:41:03.779420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.779548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.779674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.779831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:41:03.779961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.780088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.780214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.780338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.780338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.780338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.791383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.791661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.791928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.792208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.792480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.792756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.793023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.793300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.800948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.802004 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.804190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.804498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.804781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.805451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.805984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806689 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806777 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.806953 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.807032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.807217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.807479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.807745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.808009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.834491 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.835264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.836135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.836423 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.836870 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.837143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.842578 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.842852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.843677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.843837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.843973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.843973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.850381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.850653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.850797 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.850936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.855250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.855494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.855626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.855766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.855895 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.856118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.856300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.856449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.856616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.856953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857095 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857223 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857349 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.857768 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.876652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.876871 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.877003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.877149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.881583 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.881764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.885296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.885454 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.885599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.885727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.891133 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.891314 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.894701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.894851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.898393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.898682 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.898812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.898965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.899943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.900053 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.900221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.919336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.919616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.919775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.919961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.920091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.920217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.920361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.920598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.928072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.928419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.929846 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.930115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.930243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.930440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.933221 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.933499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.933664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.933795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.933948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.934155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.934363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.934496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.934808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.935086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.947952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.948234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.948493 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.948640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.948768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.948903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.949696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.950471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.950756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.968778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.969606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.972032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.973273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.975310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.976168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.976913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.977805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.978537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.978815 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.979090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.979446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.979737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.980022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.987646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.988233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.989598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.989929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:03.992553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.992880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.993117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.993386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.993490 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.993571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.993719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:03.994001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.004285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:04.004642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.004968 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.006126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:04.006417 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.006695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.013047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:04.013321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.013604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.013888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.014162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.014439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.014710 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.014972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.015237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.015323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86704) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:41:04.015513 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.015756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.015852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.015929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.016904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.017174 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.017439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.017701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.017972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.018241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.018500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.018758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86704) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:41:04.056117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558b9dc5a9c0 pid: 86703 (default app) May 19 19:41:04.060246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d07a-9789-44ff-a6df-9e5d710de8bc None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:04.060563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d07a-9789-44ff-a6df-9e5d710de8bc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:04.060846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9d07a-9789-44ff-a6df-9e5d710de8bc None None] PATH_INFO: `/v3` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:04.063326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558b9dc5a9c0 pid: 86704 (default app) May 19 19:41:04.065012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu May 19 19:41:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:12.283625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:12.284161 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:12.284161 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:12.288441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:41:12.294711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:12.295201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:12.295378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:12.363581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: INFO dbcounter [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] Registered counter for database keystone May 19 19:41:12.386491 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG oslo_db.sqlalchemy.engines [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86703) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:41:12.392890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writer thread running {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:41:12.586396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG passlib.handlers.bcrypt [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86703) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:41:12.586734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG passlib.handlers.bcrypt [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86703) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:41:12.647295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:12.648992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:12.779669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 19:41:12.957692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:13.029401 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: INFO dbcounter [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] Registered counter for database keystone May 19 19:41:13.049841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG oslo_db.sqlalchemy.engines [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86704) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:41:13.056409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writer thread running {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:41:13.184085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180c94c-f06e-4aac-acaa-29e3f3b50c81 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:13.188469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '160a68f8af7f400ab04b7ad21bdaf62d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:13.189142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:13.189426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:13.189699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:13.197979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:13.252989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: WARNING keystone.common.rbac_enforcer.enforcer [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:41:13.253395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9ea4-7518-4bd7-a2fc-467caf5dbf62 admin admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:13.267695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 19:41:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:41:13.273677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:13.289051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ed45d-0446-4b88-9313-73999266b912 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:13.293877 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '160a68f8af7f400ab04b7ad21bdaf62d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:13.294989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:13.295512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:13.295592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] PATH_INFO: `/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:13.297095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] RBAC: Authorizing `identity:get_project(project_id=160a68f8af7f400ab04b7ad21bdaf62d)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:13.466022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:41:13.466721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18dc9b1-52b9-4b84-9a58-60c78c850f44 admin admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:13.474039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu May 19 19:41:13 2022] GET /identity/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:15.723620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fbd96-2d2b-4c69-88b5-c0b562866cec None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:15.724165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fbd96-2d2b-4c69-88b5-c0b562866cec None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:15.724165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fbd96-2d2b-4c69-88b5-c0b562866cec None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:15.728301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:41:15.734501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:15.734918 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:15.735066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:15.754369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:15.755927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:15.764446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:41:15.787557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fbd96-2d2b-4c69-88b5-c0b562866cec None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:15.800272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fbd96-2d2b-4c69-88b5-c0b562866cec None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:15.804599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '160a68f8af7f400ab04b7ad21bdaf62d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:15.805149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:15.805429 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:15.805705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:15.819312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:15.821257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31017141-1f7d-4186-9bc2-1e1a30284be5 admin admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:15.831814 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 19:41:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:41:15.837626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:15.847732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852068d6-4b09-4db0-88fe-55b366a25d32 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:15.852318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '160a68f8af7f400ab04b7ad21bdaf62d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:15.853234 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:15.853511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:15.853787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:15.855413 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:15.857359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c95623-5e98-4dee-b4c4-c4c17f7a15a3 admin admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:15.858977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu May 19 19:41:15 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:18.176042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592bd9-884f-4b1a-9c36-12366e0f527a None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.176602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592bd9-884f-4b1a-9c36-12366e0f527a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.176602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e592bd9-884f-4b1a-9c36-12366e0f527a None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.177966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:18.183753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ee36-a08d-4d50-b801-37afd93cdbd7 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.184461 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ee36-a08d-4d50-b801-37afd93cdbd7 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.184461 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3ee36-a08d-4d50-b801-37afd93cdbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.201154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-35c3ee36-a08d-4d50-b801-37afd93cdbd7 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:18.202333 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-35c3ee36-a08d-4d50-b801-37afd93cdbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:18.286540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 6/11] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:18.309368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592bd9-884f-4b1a-9c36-12366e0f527a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.313334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3bf-5948-4d3a-a7d2-74b33bec5fa2 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.313630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3bf-5948-4d3a-a7d2-74b33bec5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.314000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3bf-5948-4d3a-a7d2-74b33bec5fa2 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.315890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 7/12] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:18.322003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.322292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.322576 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.340663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:18.341870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:18.358706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:18.422111 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:18.444274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.463278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e592bd9-884f-4b1a-9c36-12366e0f527a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.500003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:18.500829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.501113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.501384 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.513932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:18.525112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:18.526176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:18.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:18.532632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.567389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b2e62-644e-4ec0-b8cc-55f0ddf7bed6 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.567562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:18.610242 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:18.610906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.611180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.611453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.634807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:18.643396 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7060a-ce6f-4a50-aa8a-711b726b50ca None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.679153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:18.679825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.680111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.680392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.681178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:18.681490 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:18.683560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:18.686764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:18.688322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:18.697016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:18.703472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.793823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 9/16] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:41:18.799272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.810020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:18.816534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88515e49-b482-4d36-b7da-fcb41df069a7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.853942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:18.861917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:18.862799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.863091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.863367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.865042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:18.866504 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:18.868315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:18.873939 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:18.912434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a22286-d2a6-47bd-a139-46eb28e2bec0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.947204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:18.948311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:18.951579 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:18.951579 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:18.960432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:18.961929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:41:18.987391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:18.988458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:18.989276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:41:18.989276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:41:18.989276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:41:18.989276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:18.989276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:18.992028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050777-8398-4fc2-b9fb-2edf402bcf8d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:18.994645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:19.036904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:19.037697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:19.037973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:19.038244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:19.039507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:19.040976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:19.042727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:19.048562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:19.081076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:19.101763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b7e26-da9f-43b1-b273-cc97e3717f5a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:19.104613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:19.140904 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:19.141566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:19.141852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:19.142132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:19.142861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:19.143174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:19.144605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb00f9c-ed32-46a1-a50c-b0b7dedba369 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:19.163983 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:19.166301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea565f-9b4a-4824-b8f9-c73e216623c4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:19.222292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:19.223007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:19.223395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:19.223676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:19.224593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.domain.id=default, user.id=8197d33213ab455786d2c8bd2697b7ff) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:19.224916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:19.226441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f2192-ce54-4c7f-919e-99159fc5de18 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:19.237005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 19:41:19 2022] GET /identity/v3/role_assignments?user.id=8197d33213ab455786d2c8bd2697b7ff&scope.domain.id=default&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 233 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:21.790915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719716a7-aed1-40f5-a4f3-6238b8471e04 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:21.791435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719716a7-aed1-40f5-a4f3-6238b8471e04 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:21.791542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719716a7-aed1-40f5-a4f3-6238b8471e04 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:21.792895 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:21.798413 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:21.798850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:21.799067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:21.820390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG passlib.handlers.bcrypt [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86704) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:41:21.820838 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG passlib.handlers.bcrypt [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86704) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:41:21.869179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5cf9-c912-42f2-9fbc-0fb0a11d1e3f None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:21.869481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5cf9-c912-42f2-9fbc-0fb0a11d1e3f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:21.869732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f5cf9-c912-42f2-9fbc-0fb0a11d1e3f None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:21.871839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:41:21.878072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:21.878416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:21.878734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:21.895725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:21.897250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:21.911682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:21.913917 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:22.063471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 14/24] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:22.076901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 11/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:22.103040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:22.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:22.316191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:22.332433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:22.348152 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6919c8-51f7-461d-a07d-e8cf4d14194c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:22.443495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:22.444732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.445236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.445710 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.469873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ed5af9e3-90ee-4e8b-b407-601f3c0901aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:22.480829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:22.481763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:22.482507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:22.485536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 15/26] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:22.487301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43efcd-53fe-4afd-a160-c9383ae508fb None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.487892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43efcd-53fe-4afd-a160-c9383ae508fb None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.488448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43efcd-53fe-4afd-a160-c9383ae508fb None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.490191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 16/27] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:22.494956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43efcd-53fe-4afd-a160-c9383ae508fb None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:22.516354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5dc38-bac6-4111-bd25-fe56d26fa9bd None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:22.569317 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:22.571078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.571078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.571078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.618703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:22.620068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:22.649806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43efcd-53fe-4afd-a160-c9383ae508fb None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ae5ab55b-0659-4f1e-b5f4-0628e8f78e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:41:22.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:22.658494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:22.659378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:41:22.659378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:41:22.659378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:41:22.659378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:22.659378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 12/28] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 542 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:22.660287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0adcc-4880-4865-aef5-57fa0abfa820 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.660599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0adcc-4880-4865-aef5-57fa0abfa820 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.660877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0adcc-4880-4865-aef5-57fa0abfa820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.688474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-77f0adcc-4880-4865-aef5-57fa0abfa820 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:22.690040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-77f0adcc-4880-4865-aef5-57fa0abfa820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:22.694851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:22.695948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.696503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.697003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.698011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:22.700256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:22.701972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:22.716585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 17/29] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:22.717457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:22.737987 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:22.768061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 13/30] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:22.769739 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd5073-9b49-4941-8a15-263442b8f752 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.770123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd5073-9b49-4941-8a15-263442b8f752 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.770475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd5073-9b49-4941-8a15-263442b8f752 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.772136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 14/31] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:22.777845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd5073-9b49-4941-8a15-263442b8f752 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:22.789217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:22.839698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb747da-7321-4072-8fd3-4a39d98ff15b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:22.890151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:22.891705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.892142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.892639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.894040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:22.894501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:22.896909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24493eff-dfb2-45e2-9904-0382175080a6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:22.916090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:22.918264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.918722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.919140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.927005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd5073-9b49-4941-8a15-263442b8f752 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:22.942019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:22.943999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:22.972237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:22.973396 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:22.973868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:22.974359 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:22.976154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:22.977830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:22.980740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 15/33] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:22 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:22.981823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.043305 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 19/34] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:23.052774 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.062964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:23.069880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:23.095389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db4185-529f-4f62-8c95-ea0227b6273b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:23.137358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:23.138211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.138595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.138977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.140079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:23.141414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:23.217352 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01dc7bf-308e-4a0f-aefc-0e024016eff7 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:23.280876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:23.282211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.282859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.283531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.284977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:23.287725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:23.350000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 16/35] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:41:22 2022] POST /identity/v3/roles => generated 219 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:23.351186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.441157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 20/36] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:41:23 2022] POST /identity/v3/projects => generated 301 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:23.442433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.523728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b007ef-5022-4f4a-adc8-4781abbf3427 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:23.538696 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:23.607125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f96d65-270e-4889-930e-aa3913cfa9e2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:23.613369 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:23.614210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.614498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.614778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.616250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:23.617938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03114f5-2373-405b-bf1c-a2bec0567461 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:23.619841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 17/37] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:23.632528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbc504-3343-44d4-be14-5812dc8c98ea None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.632869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbc504-3343-44d4-be14-5812dc8c98ea None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.637042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbc504-3343-44d4-be14-5812dc8c98ea None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.637042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 18/38] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:23 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:23.637692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbc504-3343-44d4-be14-5812dc8c98ea None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.646193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:23.669670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:23.670837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.671321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.671816 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.672826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:23.674862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26136822-7440-46dd-9026-b89d609ed974 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:23.694388 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 21/39] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:41:23 2022] POST /identity/v3/roles => generated 225 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:23.696424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877656d-9e6a-4a5b-acb7-80b97c168edc None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.697353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877656d-9e6a-4a5b-acb7-80b97c168edc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.697884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877656d-9e6a-4a5b-acb7-80b97c168edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.722408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8877656d-9e6a-4a5b-acb7-80b97c168edc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:23.724870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8877656d-9e6a-4a5b-acb7-80b97c168edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:23.773406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbc504-3343-44d4-be14-5812dc8c98ea None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:23.818600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:23.819598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.819901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.820219 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] PATH_INFO: `/v3/domains/default/users/8197d33213ab455786d2c8bd2697b7ff/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.823248 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8197d33213ab455786d2c8bd2697b7ff, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:23.825641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:23.826134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 22/40] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:23.848803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66faa56-17da-4eb9-887b-c0d63c36eb55 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.849801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66faa56-17da-4eb9-887b-c0d63c36eb55 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.850254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66faa56-17da-4eb9-887b-c0d63c36eb55 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.851887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 23/41] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:23.856919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 19/42] 10.222.0.13 () {58 vars in 1361 bytes} [Thu May 19 19:41:23 2022] PUT /identity/v3/domains/default/users/8197d33213ab455786d2c8bd2697b7ff/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:23.858396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:23.858503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:23.859428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:23.872720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:23.881049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:23.881803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:23.972380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:23.983196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:24.020425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 24/43] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:24.039722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc2f169-accf-4a07-9f07-3a3473b88968 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:24.052330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:24.085005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:24.085696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:24.085990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:24.086270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:24.087035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:24.088502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:24.111713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 20/44] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:41:23 2022] POST /identity/v3/roles => generated 223 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:24.178105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815cfc5-d1ff-401c-adb5-33589f9f1fac None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:24.218943 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:24.219777 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:24.220331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:24.220459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:24.221755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:24.223249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2726c76-8e8f-4341-a236-8fa515e8ca6f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:24.224957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:24.230888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:24.362923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb899769-6b5e-4eaf-bb6a-7d6f3057dcf8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:24.389178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:24.402696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:24.403392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:24.403681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:24.403965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:24.404729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:24.406191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817e30f-a8d0-45c9-90f9-471be9bb6622 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:24.439558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:41:24 2022] POST /identity/v3/projects => generated 312 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:26.217391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224e76d-053d-411f-b642-b75caa319020 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.217930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224e76d-053d-411f-b642-b75caa319020 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.218012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224e76d-053d-411f-b642-b75caa319020 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.219405 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:26.224746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.225148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.225419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.243300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:26.245473 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:26.317385 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:26.341437 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224e76d-053d-411f-b642-b75caa319020 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:26.390960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:26.455175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224e76d-053d-411f-b642-b75caa319020 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:26.489418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:26.490246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.490540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.490828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.503207 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:26.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:26.512880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:26.513649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:26.518929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:26.632789 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4538f-fbf4-480e-ac5a-579f0cc3bca3 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:26.640785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:26.679027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:26.679697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.679984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.680265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.681046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:26.681354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:26.683572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514f789-19aa-4c64-8306-8a690216986c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:26.691991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 23/50] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:26.697482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:26.747074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d0-e623-452f-985a-c3b21a54dac4 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.747558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d0-e623-452f-985a-c3b21a54dac4 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.747897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094811d0-e623-452f-985a-c3b21a54dac4 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.749203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 24/51] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:26.754777 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.755063 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.755344 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.772021 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:26.773257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:26.814287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78d24c-ea1d-4e00-8e07-832de729ac0a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:26.821654 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:26.845679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 25/52] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:26.851660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:26.852563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:26.852851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:26.853130 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:26.858532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:26.860092 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:26.862045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 28/53] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:26 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:26.867485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:26.868368 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:26.968448 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:26.973954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0be8f3-4e52-4a6b-96e1-85541fd3b284 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:26.987748 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066fb3b-0814-4e7f-9cca-caafbfd9ac57 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:27.008469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:27.009193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:27.009472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:27.009747 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:27.011711 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:27.013235 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:27.014954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 26/54] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:27.020833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:27.032216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:27.033043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:27.033330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:27.033600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:27.034787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:27.036243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:27.037890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:27.043704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:27.111018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:27.138906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db501472-4598-40a7-b415-20226c8f4137 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:27.176273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:27.176978 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:27.177320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:27.177545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:27.178392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.domain.id=default, user.id=8197d33213ab455786d2c8bd2697b7ff) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:27.178703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:27.180165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4632a-c432-4498-b839-63f0b4a96050 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:27.180839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2cceb8-f7b5-4ed6-9e22-a9c6e10053df None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:27.187102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 27/56] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 19:41:27 2022] GET /identity/v3/role_assignments?user.id=8197d33213ab455786d2c8bd2697b7ff&scope.domain.id=default&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:27.226691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:27.227391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:27.227682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:27.227964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:27.228868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:27.230915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a1fd5-8d0e-461c-ab70-3aafcbe84223 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:27.260303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 30/57] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 19:41:27 2022] POST /identity/v3/projects => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:41:29.547222 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297bde5-00f8-441d-80ab-e5188f94298b None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:29.547743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297bde5-00f8-441d-80ab-e5188f94298b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:29.547827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297bde5-00f8-441d-80ab-e5188f94298b None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:29.549177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:29.554375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:29.554774 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:29.554936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:29.573381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:29.574742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:29.643752 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:29.669579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297bde5-00f8-441d-80ab-e5188f94298b None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:29.700012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:29.792035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297bde5-00f8-441d-80ab-e5188f94298b None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:29.828172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:29.828919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:29.829205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:29.829512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:29.830713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:29.832175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a9c6c-c031-482c-8490-b5efb65ef1e5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:29.833833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:29.839637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:29.863890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:29.945906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed5d5e-329f-440a-889b-77f7be743bce None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:29.987307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:29.987998 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:29.988285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:29.988589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:29.989317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:29.990790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b27b8d-530c-4478-a669-503d3706d05d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:30.017182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:29 2022] POST /identity/v3/users => generated 275 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:33.597298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11f68-44c9-47e8-b75b-47e50c550312 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.598312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11f68-44c9-47e8-b75b-47e50c550312 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.598796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee11f68-44c9-47e8-b75b-47e50c550312 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.600447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:33.606272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1621b-25b6-40d8-bc13-880c0faeb730 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.607272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1621b-25b6-40d8-bc13-880c0faeb730 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.607788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1621b-25b6-40d8-bc13-880c0faeb730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.630079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-27d1621b-25b6-40d8-bc13-880c0faeb730 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:33.631691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-27d1621b-25b6-40d8-bc13-880c0faeb730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:33.695051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcc29-0562-4633-ba28-d0ad36a86007 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.695676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcc29-0562-4633-ba28-d0ad36a86007 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.696147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fcc29-0562-4633-ba28-d0ad36a86007 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.698952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 31/63] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:33.704946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.705566 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.706039 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.735911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:33.737539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:33.751596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 33/64] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:33.753336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96db49-54bc-4176-8a3d-317b35311251 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.753627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96db49-54bc-4176-8a3d-317b35311251 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.753906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96db49-54bc-4176-8a3d-317b35311251 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.755731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 34/65] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:33.761481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:33.762079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:33.762535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:33.780652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:33.782142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:33.790808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:33.831293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 32/66] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:33.837521 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:33.856273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 35/67] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:33.870824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:33.898382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:33.981649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee69b2-ee68-46e6-87aa-ad072abbab2f None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.004373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8346-8989-4790-996b-5f21e3a0a9fd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.020725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.021545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.021819 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.022097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.032940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:34.040721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:34.042191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:34.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 33/68] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:34.043619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.054597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.055980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.056270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.056593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.067716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.071781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-689bb385-8563-4ece-a961-b9e78b88d8c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:34.079965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:34.080870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 36/69] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:34.086841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf6d9-d369-4d16-b8ea-10f64150fb2e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.087499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf6d9-d369-4d16-b8ea-10f64150fb2e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.087499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bf6d9-d369-4d16-b8ea-10f64150fb2e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.087499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 37/70] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:34.087499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bf6d9-d369-4d16-b8ea-10f64150fb2e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.165875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.182136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c329414-6db2-481f-aedc-173b4baf6493 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.216303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bf6d9-d369-4d16-b8ea-10f64150fb2e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.227448 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.228274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.228595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.228927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.240303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:34.256243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:34.257315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 34/71] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:34.258258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.262897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.263585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.263958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.264238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.265027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:34.265345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.266766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf69f4-8143-4fa0-ac66-412d7938cee2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.278238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 38/72] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.279521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.279942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.280158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.297851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:34.299055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:34.359790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.372078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 39/73] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:34.374379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca6380-9ed0-4920-a418-e7004e239487 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.382248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.409730 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.410420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.410707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.410985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.411748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:34.412059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.413507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.427260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 35/74] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.427260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.436092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.493937 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e009d00-9cef-476c-871a-9132d34bf10f None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.530712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.531391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.531686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.531969 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.532757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:34.533070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.534467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bdd3c-dbb4-4753-8d2f-d749a94a6347 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.538131 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c10365e-993b-4f88-bc55-215671ca23e7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.542924 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 40/75] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.544179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268853-7695-4766-b052-96696b603756 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.544501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268853-7695-4766-b052-96696b603756 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.544782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268853-7695-4766-b052-96696b603756 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.546302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 41/76] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:34.546900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268853-7695-4766-b052-96696b603756 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.581191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.582075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.582358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.582635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.583839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.585326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.586975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 36/77] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.589472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.607879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.655346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268853-7695-4766-b052-96696b603756 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.676076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.692817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.693649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.693846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.694129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.699904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2636c-e029-432d-a35b-fa405881fd8d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.705407 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:34.714530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:34.715395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:34.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 42/78] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:34.716808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.740562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.741439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.741723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.741997 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.758374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.759822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6e7d7-b094-44a2-883e-305642abc61f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.761884 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 37/79] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.763082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.763371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.763648 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.781018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:34.782240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:34.841657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a033e5-0c21-42e9-8e4c-4f7d796acfce None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.857007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.862780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 38/80] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:34.863646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.879038 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:34.880032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:34.880317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:34.880635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:34.882395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:34.885602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:34.887589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 43/81] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:34.956504 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:34.995939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae58cb89-858c-49c4-bd64-5b22f902d06c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:34.996146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0c080-8ab0-476a-8e1d-d6b7358dadcc None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.031594 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.032260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.032582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.032863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.038196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.038898 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.039191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.039467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.040238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:35.040591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.042165 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.051086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 44/82] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:35.051900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.099589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.117493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.118993 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.182729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b36243-b697-42ed-bcb1-f9bd136e94e9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.198009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.219692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.220256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.220571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.220851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.235623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 39/83] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:35.236338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.295993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.298672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.367990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.383490 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7f405-d4a7-4916-91e6-351b5ac89a2e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.420711 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.421517 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.421792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.422072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.423265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.424757 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.426378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 40/84] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:35.426378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.431178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 45/85] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:35.431383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.468515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.543110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c11705-70f5-400e-904c-3d4ff99610d3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.544669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56add7-68ed-4371-866a-c430647196fa None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.579186 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.580060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.580346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.580655 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.581864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.581964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.582540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.582823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.583099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.583316 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.584986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 46/86] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:35.585744 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.602803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.660903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.662406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.698191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36713443-bad7-4154-a767-9e5a61da792e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.699065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.734480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.735463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.735745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.736026 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.737231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.738681 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.740457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 47/87] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:35.745304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.781949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 41/88] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:35.782451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.829333 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:35.849684 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de720b53-1713-45ff-8de8-2d85d2ba6352 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.887735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2287-475c-4fec-a7e1-4e3e37ec9f22 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:35.894000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.894685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.894970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.895253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.895970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.897460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.928154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 48/89] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:35 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:35.928154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.935540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:35.936504 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:35.936794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:35.937068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:35.938427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:35.939917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:35.941927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 42/90] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:35.948321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:35.951372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.049125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494f025-9ba7-4c94-b145-c1c7ac8f5576 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.095718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.096320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.096657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.096941 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.099111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14ae09-c1d9-4aa2-89e5-4b506eefe99f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.100989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.154178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.154852 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.155138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.155409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.156457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=8197d33213ab455786d2c8bd2697b7ff) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:36.156679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.158148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.165418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 43/91] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/role_assignments?user.id=8197d33213ab455786d2c8bd2697b7ff&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:36.166435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:36.202351 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.203933 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.262048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.317052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea649726-83a1-4782-9986-07a1a49a3477 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.351770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 49/92] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:36.354219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:36.360245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.361181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.361467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.361744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.363044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.365224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.371149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 44/93] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:36 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:36.371149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:36.412984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.512117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd79be6-78a2-45eb-a350-4515a2ad17c9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.535867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953397f-5d80-4649-9062-88d50e63a79f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.569197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.570006 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.570406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.570781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.571723 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:36.572134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.574348 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.581883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:36 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:36.582805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:36.602314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.603151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.603436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.603811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] PATH_INFO: `/v3/projects/dd65fa7108424da59ac816373c645f1e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.609506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] RBAC: Authorizing `identity:get_project(project_id=dd65fa7108424da59ac816373c645f1e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.611330 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.613324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 45/95] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:36 2022] GET /identity/v3/projects/dd65fa7108424da59ac816373c645f1e => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:36.624480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:36.646883 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.816540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3728168-1721-473e-98c9-61c2624f3548 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.825357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:36.827633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b32f0-0d3e-4677-a683-f88c6ca210ee None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:36.869170 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.869881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.870182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.870459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.871434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:36.872012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.880601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf795f-9cfe-4b2b-b1f9-9c9892d6262f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.880601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:36 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:36.894436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:36.895419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:36.895862 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:36.896285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:36.897320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=dd65fa7108424da59ac816373c645f1e, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:36.897805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:36.899474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6ea63-cebd-48ae-978e-4af8f52049df None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:36.907774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:36 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=dd65fa7108424da59ac816373c645f1e&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:38.649384 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311781-eab7-4a47-a533-a3af3e63f4fd None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:38.649921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311781-eab7-4a47-a533-a3af3e63f4fd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:38.650001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee311781-eab7-4a47-a533-a3af3e63f4fd None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:38.651639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:38.670422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:38.671270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:38.671707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:38.695578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:38.697059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:38.766263 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:38.774504 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:38.824986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311781-eab7-4a47-a533-a3af3e63f4fd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:38.941253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee311781-eab7-4a47-a533-a3af3e63f4fd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:38.979503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:38.980318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:38.980637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:38.980921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:38.993085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:39.001486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:39.002509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:39.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 53/100] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:39.009867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:39.118671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2e265-7e08-4432-b1d8-7ba57a67245d None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:39.156340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:39.157117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:39.157404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:39.157731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:39.158499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:39.158809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:39.160199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:39.170796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 48/101] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:39.176126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:39.209620 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:39.294889 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25166687-877e-4bf2-a85e-036e25eac722 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:39.336457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:39.337695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:39.338455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:39.338911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] PATH_INFO: `/v3/users/251d1f23a37047aea8c72150af25c9b3` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:39.355155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] RBAC: Authorizing `identity:get_user(user_id=251d1f23a37047aea8c72150af25c9b3)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:39.356971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:39.358950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 54/102] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:39 2022] GET /identity/v3/users/251d1f23a37047aea8c72150af25c9b3 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:39.364802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:39.444940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:39.502692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a0e6d-9742-47f2-ad58-7f25487a836a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:39.558028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:39.558974 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:39.559428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:39.559861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:39.656741 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:39.658492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:39.740221 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:39.802389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 49/103] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:39.809075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:39.921679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:39.952241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee26b6-f0e5-42b3-9269-9e68bff2c125 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.007995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.008924 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.009212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.009635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.010842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:40.012518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a46612-275c-4c93-bebd-e76ea1ee86ff None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:40.014434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 55/104] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:39 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:40.020537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.147315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c882-9e52-4277-a4c7-d85a934bcae3 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.147604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c882-9e52-4277-a4c7-d85a934bcae3 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.147887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003c882-9e52-4277-a4c7-d85a934bcae3 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.149633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 56/105] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:40.158039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.158363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.158613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.164006 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ebdd5-30a5-40b1-98db-9e3abde36702 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.199801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:40.201043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:40.202121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.202780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.203139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.203419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.204252 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=251d1f23a37047aea8c72150af25c9b3) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:40.204591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:40.206070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed54668-fcde-47dc-9025-d1b1ae2feae8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:40.212664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 50/106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/role_assignments?user.id=251d1f23a37047aea8c72150af25c9b3&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:40.225528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a51c-d7bf-4052-8770-f41135b3efa0 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.225864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a51c-d7bf-4052-8770-f41135b3efa0 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.226093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57a51c-d7bf-4052-8770-f41135b3efa0 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.227769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 51/107] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:40.234661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.235275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.235765 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.257962 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:40.259249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:40.282209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:40.329753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 57/108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:41:40.352908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.381356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 52/109] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:40.418882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.442963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:40.471382 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba2770-5559-4c76-a58f-591215831b99 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.515180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.516026 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.516312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.516622 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.529296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-90574b9a-9e17-4c8c-9c11-d897655eeab5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:40.537730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:40.538628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:40.539373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 58/110] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:40.544799 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc9d-1fe7-445f-8aaa-0b2e58fc6859 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.545135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc9d-1fe7-445f-8aaa-0b2e58fc6859 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.545437 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193bc9d-1fe7-445f-8aaa-0b2e58fc6859 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.547224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 59/111] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:40.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b37b-5678-4923-a17f-3b300d19fc44 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b37b-5678-4923-a17f-3b300d19fc44 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0b37b-5678-4923-a17f-3b300d19fc44 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 60/112] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:40.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0b37b-5678-4923-a17f-3b300d19fc44 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.594406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee9f02-8152-4c22-9cb8-a694e0973183 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.634918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.636028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.636155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.636354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.647474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-410e3b28-b301-4f29-8a6e-a2c7dbae34d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:40.655507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:40.656447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:40.657341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 53/113] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:40.658048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3128-c400-448a-a241-0a391303c74a None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.658930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3128-c400-448a-a241-0a391303c74a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.658930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3128-c400-448a-a241-0a391303c74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.678171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0b37b-5678-4923-a17f-3b300d19fc44 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.679637 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-3a9b3128-c400-448a-a241-0a391303c74a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:40.680867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3a9b3128-c400-448a-a241-0a391303c74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:40.689680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:40.717912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.718604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.718894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.719244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.719941 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:40.720257 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:40.722672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd9235-4200-41ac-a7cf-bb82421f9c36 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:40.732742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 61/114] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:40.734072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.734361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.734635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.739815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:40.758975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:40.760432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:40.768472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 54/115] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:40.774635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c270f-c9d1-4dba-8b8d-db4eae2309ea None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.775194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c270f-c9d1-4dba-8b8d-db4eae2309ea None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.775677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c270f-c9d1-4dba-8b8d-db4eae2309ea None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.778030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 55/116] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:40.779276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c270f-c9d1-4dba-8b8d-db4eae2309ea None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.844261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 62/117] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:40.844982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:40.904782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c270f-c9d1-4dba-8b8d-db4eae2309ea None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.943793 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:40.944375 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.944682 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.944961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.945712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:40.946014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:40.947475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce66aee-a086-4482-8437-721ec637e3c0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:40.960487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 56/118] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:40.960487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:40.960487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:40.960487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:40.970070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:40.974829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaae429-61c9-40ad-b4e3-83578240955a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:40.984674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:40.986333 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:41.011102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.017445 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.018158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.018451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.018753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.020046 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.021587 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.023588 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 63/119] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:40 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.025472 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.063509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 57/120] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:41.063509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.155212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6ee73-7092-4433-896c-158802f61852 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf903-30ce-4353-8127-7dc616dfdb09 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.196289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.197320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.197741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.198184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.199505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.201139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.203216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 64/121] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.204117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.231973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.232868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.233156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.233611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.235817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.247328 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:41.255239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:41.256052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:41.256977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 58/122] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:41.262270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.282369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.319783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385a5f4-06b5-48b8-89f9-37bd769914a8 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.358229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.359264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.359547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.359820 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.361048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.362504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.364348 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 65/123] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.365031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.386255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9ae3-2826-40bb-a302-18115a10ddd6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.429375 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.430257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.430551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.430833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.432298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.433822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.435736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 59/124] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.436344 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.487959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21ab94-9c99-4ab2-be7f-135619c11ac7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.514731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.528497 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.529308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.529598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.529876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.533406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.544983 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:41.553154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:41.554108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 66/125] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:41.559555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.569928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ea861-fbef-43b3-a85d-96116575e01e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.610965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.611750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.612033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.612307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.613177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.614625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.643790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 60/126] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:41:41 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:41.649621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.680599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b508e-77b8-4b2c-9d92-a7a99b2c0abe None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.723341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.724068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.724354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.726728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.727065 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:41.727378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.729533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.739794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 67/127] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.740583 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.799382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022ec86-f128-4dc7-858e-ad2c6c4050f7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.816797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.825812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:41.853056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.854574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.854702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.855266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.856566 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.858102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.860971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 61/128] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:41.861486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:41.874770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf58d5-ecae-479e-8646-a6d1d53a3869 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:41.916819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:41.918006 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:41.918288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:41.918565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/a66e55122f6a41078aeed1852f492fe7/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:41.921114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, user_id=a66e55122f6a41078aeed1852f492fe7, role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:41.922803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:41.941866 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 68/129] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:41 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/a66e55122f6a41078aeed1852f492fe7/roles/afe01609a07c422aa82e18ea2bb45744 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:41.942516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.010983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b762d63-7e54-4872-b95d-82519afe17df None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.097254 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.098450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.099821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.100967 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.104774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:42.105334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.106978 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.118206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 62/130] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:42.120553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.190624 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833e50-aa4e-4ef4-8e57-9fb2cda58159 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.200725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:42.221304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:42.246271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.247408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.247694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.247967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.249250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.250818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.252946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 69/131] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:41 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:42.253880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.277604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729bda-52ec-4546-aa7c-c9a42eb41345 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.319820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac521-4857-4686-8511-910188672644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.321106 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac521-4857-4686-8511-910188672644 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.321623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac521-4857-4686-8511-910188672644 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.322715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dac521-4857-4686-8511-910188672644 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/8197d33213ab455786d2c8bd2697b7ff/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.325017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dac521-4857-4686-8511-910188672644 None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, user_id=8197d33213ab455786d2c8bd2697b7ff, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.326710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dac521-4857-4686-8511-910188672644 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.344674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 63/132] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:42 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/8197d33213ab455786d2c8bd2697b7ff/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:42.345996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac521-4857-4686-8511-910188672644 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.425327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018e2b3-7be6-40aa-8250-1819d954332d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.475649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dac521-4857-4686-8511-910188672644 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.479207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:42.498969 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.499799 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.500082 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.500362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.505234 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.506781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.508928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 70/133] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:42 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:42.517504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.521758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.522979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.523515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.524007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] PATH_INFO: `/v3/projects/dd65fa7108424da59ac816373c645f1e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.526744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] RBAC: Authorizing `identity:get_project(project_id=dd65fa7108424da59ac816373c645f1e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.528641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.530736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 64/134] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:42 2022] GET /identity/v3/projects/dd65fa7108424da59ac816373c645f1e => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:42.543291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.656862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:42.723091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4fd0-09c3-44b7-a88c-a231381dfb7c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.779732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.780601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.780897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.781184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:42.782526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:42.784659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:42.786564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 71/135] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:42 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:42.793675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:42.875268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2c705-c49d-4acd-b618-3441260d0d0c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.984251 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098136-87e4-4c30-b11a-10b3ca32ae93 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:42.987764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:42.989190 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:42.989474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:42.989754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] PATH_INFO: `/v3/projects/dd65fa7108424da59ac816373c645f1e/users/a66e55122f6a41078aeed1852f492fe7/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.001494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] RBAC: Authorizing `identity:create_grant(project_id=dd65fa7108424da59ac816373c645f1e, user_id=a66e55122f6a41078aeed1852f492fe7, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:43.003703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f5ea7-022a-4703-aa5e-42a2adcbe84d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:43.036194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 65/136] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:42 2022] PUT /identity/v3/projects/dd65fa7108424da59ac816373c645f1e/users/a66e55122f6a41078aeed1852f492fe7/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:43.037801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622bb0-4ca4-4954-b441-5fc2e4fa9f85 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:43.040146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622bb0-4ca4-4954-b441-5fc2e4fa9f85 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:43.041520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d622bb0-4ca4-4954-b441-5fc2e4fa9f85 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.043452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 66/137] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:43.051470 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:43.058285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:43.059296 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:43.060026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.101366 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:43.102680 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:43.103827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:43.104664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/a66e55122f6a41078aeed1852f492fe7/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.117845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:43.120452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:43.120849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, user_id=a66e55122f6a41078aeed1852f492fe7, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:43.122760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:43.155390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 72/138] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:42 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/a66e55122f6a41078aeed1852f492fe7/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:43.185051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:43.272256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 67/139] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:41:43.299254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:43.452025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1b294-19fd-4939-b0a7-f1701a349a18 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:43.513353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:43.514176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:43.514464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:43.514744 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.535916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:43.544075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:43.544981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:43.545717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 73/140] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:43.555211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:43.746093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a4331-6c73-4b89-ba4d-b7a04a3b193d None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:43.796903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:43.797611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:43.797901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:43.798182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:43.798977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:43.799295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:43.800778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:43.812961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 68/141] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:43.818696 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:43.881307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:43.963445 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc91b99-2f12-4b59-ad7f-f63da84a12f3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:44.046569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:44.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:44.047741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:44.048012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] PATH_INFO: `/v3/users/251d1f23a37047aea8c72150af25c9b3` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:44.049307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] RBAC: Authorizing `identity:get_user(user_id=251d1f23a37047aea8c72150af25c9b3)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:44.051172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:44.052929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 74/142] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:43 2022] GET /identity/v3/users/251d1f23a37047aea8c72150af25c9b3 => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:44.058678 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:44.105942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:44.228328 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aa0a1-14c3-4eba-8c15-baa286216257 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:44.273330 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:44.274212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:44.274498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:44.274897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:44.276690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:44.278159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf153fb-cf66-444a-a509-8b7c38e544a9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:44.279883 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 69/143] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:44 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:44.285802 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:44.447732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbf42-58c9-4d1c-b0ba-2e7b181cd395 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:44.462178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:44.501236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:44.502138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:44.502416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:44.502687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/251d1f23a37047aea8c72150af25c9b3/roles/a677fb6c90ac4ea2ae46d7104319ee06` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:44.509221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, user_id=251d1f23a37047aea8c72150af25c9b3, role_id=a677fb6c90ac4ea2ae46d7104319ee06)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:44.510691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb151e92-a5cc-4a76-984d-8d526cb44850 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:44.537854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 75/144] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:44 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/users/251d1f23a37047aea8c72150af25c9b3/roles/a677fb6c90ac4ea2ae46d7104319ee06 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:45.267137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa92a-b1e2-4359-8b49-f3af84f6d168 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.267639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa92a-b1e2-4359-8b49-f3af84f6d168 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.267639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa92a-b1e2-4359-8b49-f3af84f6d168 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.269335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 70/145] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:45.275362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390105-50e5-48d9-85c9-471ca3759736 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.276013 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390105-50e5-48d9-85c9-471ca3759736 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.276013 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9390105-50e5-48d9-85c9-471ca3759736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.301702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-e9390105-50e5-48d9-85c9-471ca3759736 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:45.303056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-e9390105-50e5-48d9-85c9-471ca3759736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:45.395555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 76/146] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:45.431012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa92a-b1e2-4359-8b49-f3af84f6d168 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:45.473563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb12d8-0e53-48e4-9660-9dee57c2aeb5 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.473867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb12d8-0e53-48e4-9660-9dee57c2aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.474122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb12d8-0e53-48e4-9660-9dee57c2aeb5 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.475505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 77/147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:41:45.483176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.483467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.483741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.514339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:45.515584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:45.643393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:45.673568 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa92a-b1e2-4359-8b49-f3af84f6d168 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:45.700444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 78/148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:41:45.729477 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:45.730313 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.730595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.730869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.748474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:45.766261 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4e695736-3b17-4456-8a35-debb04fe67a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:45.777469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:45.778938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 71/149] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:41:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:45.780267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575127a7-1665-40fd-98d8-5de57d7391b1 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:45.782470 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575127a7-1665-40fd-98d8-5de57d7391b1 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:45.782470 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575127a7-1665-40fd-98d8-5de57d7391b1 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:45.782728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 72/150] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:45.787679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575127a7-1665-40fd-98d8-5de57d7391b1 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:45.804293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:45.933103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bdded-42a1-4eb9-b9dc-ae912a6c4873 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.019854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.020694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.020976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.021248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.039125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-261a7538-a444-44d1-894c-669d955f9abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:46.047329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:46.048179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:46.048945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 79/151] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:46.050121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.050121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.050286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575127a7-1665-40fd-98d8-5de57d7391b1 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.050451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:46.080314 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:46.099892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.100590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.100865 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.101177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.101881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:46.102183 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.103735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.120295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 73/152] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:41:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:46.122820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.159208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.233031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 80/153] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:46.241387 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.276051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.371966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92d33b-e9db-4835-ae61-732f06ef22f9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.391578 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dc18a-5134-4055-8336-8b1a89a6b347 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.451748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.452751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.453186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.453606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.454529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:46.455917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.457526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.465029 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.469250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.469755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.470173 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.471861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.474547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.476596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 81/154] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:46.485164 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.485359 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 74/155] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:46.486375 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.616927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.628646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.633450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5285db-623e-423b-8a55-18a5c03a8de6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.647405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c5c76-650a-40ca-b25e-bc4ebc944b92 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.688214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.689390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.689867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.690422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.692044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.693971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.696414 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 82/156] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:46.696563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.697438 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.697718 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.698000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.698519 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.798226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.800031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.834158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0615448-5341-47fb-880f-eb2ce01fe02b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:46.883079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:46.886055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.886644 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.886920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] PATH_INFO: `/v3/users/8197d33213ab455786d2c8bd2697b7ff` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.888931 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] RBAC: Authorizing `identity:get_user(user_id=8197d33213ab455786d2c8bd2697b7ff)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:46.890695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edd6de-c034-41d7-870f-84b1f6f87040 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:46.893801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 83/157] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/users/8197d33213ab455786d2c8bd2697b7ff => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:46.895040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15131d-cb0a-4e66-9052-60ddebc8a707 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:46.895472 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15131d-cb0a-4e66-9052-60ddebc8a707 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:46.895873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15131d-cb0a-4e66-9052-60ddebc8a707 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:46.897276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 84/158] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:46.898083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15131d-cb0a-4e66-9052-60ddebc8a707 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:46.910289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.934414 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:46.948683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 75/159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:46.949836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.038043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15131d-cb0a-4e66-9052-60ddebc8a707 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.066596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efdaaa-79cf-46b3-8f92-728acb74961e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.074862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.075545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.075823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.076092 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.076919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:47.078359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693115e0-300f-470a-9978-9005bb2cec96 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:47.101610 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 85/160] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:46 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:47.103516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe378ac-69ac-4dca-833b-6b4ce9c0b50a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.103846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe378ac-69ac-4dca-833b-6b4ce9c0b50a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.104338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe378ac-69ac-4dca-833b-6b4ce9c0b50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.114908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.116386 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.117104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.117591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.132996 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-4fe378ac-69ac-4dca-833b-6b4ce9c0b50a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:47.135034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-4fe378ac-69ac-4dca-833b-6b4ce9c0b50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:47.189866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.221735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 86/161] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:47.228635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f46e-1a7e-4f6b-8154-51ac7cf9043e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.228635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f46e-1a7e-4f6b-8154-51ac7cf9043e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.228635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b25f46e-1a7e-4f6b-8154-51ac7cf9043e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.228635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 87/162] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:47.231133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f46e-1a7e-4f6b-8154-51ac7cf9043e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.233620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:47.236707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccac2c1-13bd-49c9-b071-4490b8b6cd25 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:47.249019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.383337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b25f46e-1a7e-4f6b-8154-51ac7cf9043e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.393858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 76/163] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:47.396565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.397034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.397977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.420220 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:47.421641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:47.427928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.428839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.429122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.429399 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.432084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:47.433611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:47.436551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 88/164] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:47.444204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.529888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 77/165] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:47.529888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.550833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.553784 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.583250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632a353-7492-40ab-b6cc-a64de91c18e0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.630285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.630675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.630956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.631227 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.643375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:47.652129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:47.653099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:47.653847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 89/166] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:47.654743 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.694116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dad73a-7220-4713-a113-807e444c7710 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.742995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.744217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.749227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.749565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.751074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:47.752578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:47.754550 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 78/167] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:47.756111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.795803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89384240-b57e-43a6-87f6-a99353612ea3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.851436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.852483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.853883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.854402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.855502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:47.856015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:47.858202 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.859193 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dcad35-17d8-410a-b382-39ea75eced74 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:47.870775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 90/168] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:47.873497 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1da4-9243-43fb-b539-fa268079fc85 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.874027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1da4-9243-43fb-b539-fa268079fc85 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.874733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b1da4-9243-43fb-b539-fa268079fc85 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.876980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 91/169] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:47.878024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b1da4-9243-43fb-b539-fa268079fc85 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:47.885001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:47.898980 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f0de3f-6aa8-467b-a723-4a1f96e255cb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:47.954520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:47.956428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:47.956930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:47.957403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:47.973569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:47.983442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:47.984389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:47.985232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 79/170] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:47.985862 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.123951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48015faf-9ee3-48a0-854f-3461a9243457 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.125552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b1da4-9243-43fb-b539-fa268079fc85 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.166225 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.166954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.167237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.167506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.172461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=8197d33213ab455786d2c8bd2697b7ff) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:48.172461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.172461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411caec-bc56-4a97-ad37-32ba60e194fe None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.172915 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.184432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 80/171] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/role_assignments?user.id=8197d33213ab455786d2c8bd2697b7ff&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:48.185660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.186051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.186354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.217468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:48.218648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.219719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.221482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:48.222161 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.222705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.223785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:48.227014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.237922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.262585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 92/172] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:48.265457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.398740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 81/173] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:48.402551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.420880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.500664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d319ddc-f70e-4de3-8632-6b16bfee6ca6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.545567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.546587 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.547079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.547582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.548612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:48.549124 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.550726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.561350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 93/174] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:48.562556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.573357 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58778c34-1e2f-4333-972c-5aef7b2b7944 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.623288 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.634665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.635950 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.636560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.637039 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.638922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.640749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.642922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 82/175] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:48.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.710096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8657c9-9e28-41ea-bc29-d1d134267315 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.744879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:48.758397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.759404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.759692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.759965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.782820 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:48.797884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:48.798828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:48.799586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:48.801233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 94/176] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:48.810886 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:48.865383 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e83c-2444-423b-8ed8-3a6bb90cbda0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:48.954297 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:48.955557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:48.956068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:48.956573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] PATH_INFO: `/v3/users/a66e55122f6a41078aeed1852f492fe7` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:48.958019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] RBAC: Authorizing `identity:get_user(user_id=a66e55122f6a41078aeed1852f492fe7)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:48.959686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:48.961655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 83/177] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/users/a66e55122f6a41078aeed1852f492fe7 => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:48.963456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.017241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2872d-9ca0-496a-a652-0af0a3f672c9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:49.080296 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:49.087691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:49.088764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:49.089260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:49.089734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:49.137497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eb633-282d-414f-9918-32aa0676e27c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:49.175319 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:49.175989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:49.176276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:49.176580 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:49.180422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:49.180422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:49.180422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:49.188781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 84/178] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:49.189671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.233781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:49.256626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:49.258410 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:49.383095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 95/179] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:49.385019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.409280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4745ab-646e-4475-b66f-8a586799bdb2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:49.458944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:49.499787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:49.500366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:49.501243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:49.501525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:49.523954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:49.555370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b582efb-82ca-44be-8056-4c8a2e4de4e7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:49.606956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:49.608264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:49.608796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:49.609278 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] PATH_INFO: `/v3/users/251d1f23a37047aea8c72150af25c9b3` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:49.610977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] RBAC: Authorizing `identity:get_user(user_id=251d1f23a37047aea8c72150af25c9b3)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:49.613907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:49.616025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 96/180] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:49 2022] GET /identity/v3/users/251d1f23a37047aea8c72150af25c9b3 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:49.619254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.640564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:49.642424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:49.816144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 85/181] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:49.817915 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.852057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:49.870581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a23ed-d6f5-4838-a770-7d9c723ecb31 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:49.964847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:49.965733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:49.966020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:49.966299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] PATH_INFO: `/v3/projects/dd65fa7108424da59ac816373c645f1e` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:49.972421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] RBAC: Authorizing `identity:get_project(project_id=dd65fa7108424da59ac816373c645f1e)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:49.972421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:49.976575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 97/182] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:49 2022] GET /identity/v3/projects/dd65fa7108424da59ac816373c645f1e => generated 312 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:49.978509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:49.986910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ee198-1e72-4a2d-902d-00de1927cb7a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.031611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.032290 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.032600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.032877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.050168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.180702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:50.182855 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adc5a2-12eb-4f04-9cee-8a42d0659600 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:50.221409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58878dd3-040c-42a0-8210-516e2cea8982 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.233154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.272725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.273876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.274162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.274976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.276598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:50.279059 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:50.280851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 98/183] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:49 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:50.284957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:50.329299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 86/184] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:50.330639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5d40-1818-4e7f-bad6-85c81139bd30 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.330923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5d40-1818-4e7f-bad6-85c81139bd30 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.331193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc5d40-1818-4e7f-bad6-85c81139bd30 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.332482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 87/185] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:50.333171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc5d40-1818-4e7f-bad6-85c81139bd30 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:50.417022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.435977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a804-c6e7-447f-a5f6-6f2d05a33dd3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.476430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc5d40-1818-4e7f-bad6-85c81139bd30 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.481121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.481801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.482075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.482343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.483191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=dd65fa7108424da59ac816373c645f1e, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:50.483501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:50.485031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07347c-c61a-4ea5-b01c-701a5646bdd6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:50.491859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 99/186] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:50 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=dd65fa7108424da59ac816373c645f1e&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:50.498137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757be9-d0be-43ca-a83f-588facc6e549 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.498448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757be9-d0be-43ca-a83f-588facc6e549 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.498702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757be9-d0be-43ca-a83f-588facc6e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.513373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.524271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-05757be9-d0be-43ca-a83f-588facc6e549 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:50.527027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-05757be9-d0be-43ca-a83f-588facc6e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:50.528488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.529244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.529528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.529838 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.530979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=a66e55122f6a41078aeed1852f492fe7) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:50.531299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:50.533635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:50.540745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 88/187] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:50 2022] GET /identity/v3/role_assignments?user.id=a66e55122f6a41078aeed1852f492fe7&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:50.541742 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:50.673996 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 100/188] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:50.681274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6bf6a-44a7-435c-9974-1d421d5f7060 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.681712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6bf6a-44a7-435c-9974-1d421d5f7060 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.681813 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6bf6a-44a7-435c-9974-1d421d5f7060 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.683327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 101/189] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:50.684648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b17af-0893-423e-a2a8-a3ea080d4d73 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.684934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b17af-0893-423e-a2a8-a3ea080d4d73 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.685273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b17af-0893-423e-a2a8-a3ea080d4d73 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.695394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 102/190] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:50.701342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.701641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.701916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.720848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:50.722168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:50.733695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1468b7-462d-4530-b030-0f2d28f5b9b1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.788092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.789125 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.789438 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.789712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.791264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:50.792854 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff32df1-1e36-498b-9e96-db5d96e436bd None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:50.796074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 89/191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:50 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:50.797473 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.797760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.798036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:50.822617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:50.823829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:50.830001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 103/192] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:50.838024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:50.856779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.890857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:50.908102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 90/193] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:50.909020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:50.955287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d60e5-b301-4b4b-8a77-f46ad8962d17 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:50.992924 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:50.993896 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:50.994311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:50.994585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.005239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:51.013679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:51.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:51.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 104/194] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:51.016019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.030384 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bb198-4b22-4fe4-9af3-8d2dc1f01dcc None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.070620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.071822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.072108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.073331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, scope.project.id=757591eb90474b5ba0ad93f33b0a9889, user.id=251d1f23a37047aea8c72150af25c9b3) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:51.073638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.075079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.081135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 91/195] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:50 2022] GET /identity/v3/role_assignments?user.id=251d1f23a37047aea8c72150af25c9b3&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.082110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.101747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.146419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc6f3e-40d5-45bd-9636-b8a49773ee0f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.151360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.187970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.188847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.189122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.189395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.202660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:51.211120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:51.212571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:51.213400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:51.214003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aa65c-9637-40ac-b723-c44f30a07425 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.218188 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 105/196] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:51.219624 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.255417 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.256237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.256560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.256832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.258068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.259605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.261492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 92/197] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.267063 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.362223 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9329a-21cc-4b98-bee9-0bc87577d24b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.389223 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571ebae-1a3d-4b15-b58d-6cd4cffd975b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.399002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.400228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.400545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.400823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.401604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:51.401915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.403356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.406067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.411345 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.414420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 106/198] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.416056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.430864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.431532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.431804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.432074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.432853 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:51.433154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.434525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.442467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 93/199] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.443248 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.521073 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47474bec-f6cb-4861-9135-0b5ff5a07451 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.548957 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7924e5-f89d-451e-950a-cfe6fd1d3422 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.555228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.556189 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.556739 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.556837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.557498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.558976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.580045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 107/200] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:51 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:41:51.584979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.585416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.585876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.586147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.586417 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.605227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.616314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.618966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 94/201] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.624926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.672065 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.674840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.738958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f79533-207c-4d3c-9e58-1674830a1e03 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.745486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39804857-3576-41a4-9910-5f73e442f695 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:51.781517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.782323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.783629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.783629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.784059 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.785602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.787488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 108/202] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:51.792432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:51.793169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:51.793539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:51.793664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:51.793822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:51.881174 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:51.882291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:51.941390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:51.963492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652fd2d-5495-43db-84bd-d2c027547cff None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:52.002342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:52.021687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:52.024100 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:52.024732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:52.025213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:52.039467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 95/203] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:41:52.046755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:52.120117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:52.122832 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:52.182099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968bdda-48ed-4cba-bd09-c0b1f9d0e00e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:52.228739 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:52.230043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:52.230318 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:52.230589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:52.232462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:52.234272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:52.240441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 96/204] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:52 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:52.244645 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:52.258452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:52.265958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 109/205] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:41:52.266967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:52.291018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:52.379184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b1262-6116-4343-907a-e27606339cd0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:52.387982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b9ca9-f9af-4104-b5b1-a6249ac2fbfd None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:52.428708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:52.428919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:52.429619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:52.430019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:52.430343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:52.430472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:52.431414 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:52.431852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:52.431971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=94f1d28dd2924962b657e4d9daf4240e) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:52.432337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:52.433549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:52.433948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af719169-02b5-4304-ad35-128227b5b27b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:52.435374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:52.438418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 110/206] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:52 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:52.441559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 97/207] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:52 2022] GET /identity/v3/role_assignments?user.id=94f1d28dd2924962b657e4d9daf4240e&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:52.449671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:52.559203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:52.576032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba207424-0278-49f6-8365-4845f3026520 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:52.624417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:52.625105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:52.625389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:52.625665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:52.626526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=94f1d28dd2924962b657e4d9daf4240e) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:52.626843 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:52.628364 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59581925-7ac7-44ef-b386-dac449b05d4f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:52.636240 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 111/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:52 2022] GET /identity/v3/role_assignments?user.id=94f1d28dd2924962b657e4d9daf4240e&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:54.090379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46845396-1cae-46af-afb0-ef8279544b43 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.090927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46845396-1cae-46af-afb0-ef8279544b43 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.090927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46845396-1cae-46af-afb0-ef8279544b43 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.092674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 98/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:54.097977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.098375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.098558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.115300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99c37-5f74-49a3-8e59-17c324755060 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.115614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99c37-5f74-49a3-8e59-17c324755060 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.115852 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99c37-5f74-49a3-8e59-17c324755060 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.118316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 99/210] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:41:54.122907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:54.124114 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.124419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.124701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.125034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:54.146226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:54.147412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:54.198099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 112/211] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:54.226345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.243287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 100/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:54.269878 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.272340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:54.339492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:54.366153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98407d1e-59a9-46ed-9f57-1a1711f87775 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:54.406414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce5563-bbbf-4fca-84d8-df189ad30db3 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:54.413182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:54.414297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.414916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.416649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.432551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:54.444208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:54.445109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:54.445864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:54.453334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 113/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:54.453992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:54.454785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.455064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.455335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.456797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.457471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:54.459144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:54.464513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 101/214] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:54.467735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.554756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:54.568084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9892-1d80-4d1e-acb7-1c3e4952bfe2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:54.595631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57212e29-e2f1-4f71-8993-3c15641cd739 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:54.610014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:54.610819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.611074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.611304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.612079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:54.612420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:54.614206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:54.628438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 114/215] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:54.631473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.637913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:54.646965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:54.647644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.647939 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.648186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.649286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:54.651589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a198e-5832-411d-9848-280e7702b6d8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:54.692985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 102/216] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:54 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:41:54.769594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b229d-b220-4def-8ef4-173f199e198e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:54.799285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a889-8bde-41e8-b433-762841764680 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.799966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a889-8bde-41e8-b433-762841764680 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.800257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a889-8bde-41e8-b433-762841764680 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.802173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 103/217] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:54.809074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.809784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.810266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.815124 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:54.816049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:54.816329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:54.816639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] PATH_INFO: `/v3/users/53546c7e89274cea9c4d43b0f97bc8db` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:54.830317 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:54.831876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:54.832389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] RBAC: Authorizing `identity:get_user(user_id=53546c7e89274cea9c4d43b0f97bc8db)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:54.833990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:54.836074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 115/218] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/users/53546c7e89274cea9c4d43b0f97bc8db => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:54.845551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.919202 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 104/219] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:54.920415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:54.942466 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:54.972827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e0345-ae14-466c-afdc-3a3562a84075 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.017134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.017790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.018072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.018347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.022197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.080341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f06e0-d2e9-48bf-b6d1-db7ab8559733 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.113613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.115434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213fe8c-06c2-4877-95a5-ca147d56bd3f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.131389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.132857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.133434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.134879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.147030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-de6d8e98-d028-4b2c-9aaa-f459b3e90b32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:55.160587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:55.161684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:55.162657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:55.164142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 105/220] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:55.165674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5719-7dc9-4527-96d9-ef975744ce78 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.168798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5719-7dc9-4527-96d9-ef975744ce78 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.170460 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5719-7dc9-4527-96d9-ef975744ce78 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.172024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 106/221] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:55.173107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5719-7dc9-4527-96d9-ef975744ce78 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.197146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.256550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 116/222] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:55.258800 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.259131 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.259489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.276730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:55.277990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:55.312982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5719-7dc9-4527-96d9-ef975744ce78 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.332934 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.354188 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.355217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.355743 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 117/223] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:55.355946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.356637 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.356739 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.357752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:55.358327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.360064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.371145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 107/224] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:55.381708 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.474609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.485293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67461b-4cbb-4d56-880e-b313dcd77129 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.533463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.534316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf50a1-d894-45ba-abed-efacd1b399bf None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.534636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.535012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.535381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.536778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.543004 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.545219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 118/225] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:55.545829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.579307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.581034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.581422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.581798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.584039 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.585652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.589706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 108/226] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:55.590582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.662750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.670241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22750073-4a6c-4560-a853-6174af832c96 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.718424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.718424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.718424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.718787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.731908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:55.740717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:55.741605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:55.742360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 119/227] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:55.743803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.753995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7120e-d897-4ce1-8336-5dc7c754f6f9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.768825 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.795115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.795785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.796067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.796471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.797318 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=53546c7e89274cea9c4d43b0f97bc8db) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:55.797633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.799211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.806091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 109/228] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/role_assignments?user.id=53546c7e89274cea9c4d43b0f97bc8db&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:55.807116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:55.897713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9929a5d-8a3a-416b-8b34-c80d552a0aba None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.937171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d36166-6e99-49fe-a1ec-23f9316152a2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:55.953923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:55.981723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.981723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.981723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.982072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:55.982731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:55.983159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:55.984827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:55.995518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 110/229] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:55.997203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:55.998082 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:55.998360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:55.998629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:56.000795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:56.002262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:56.004058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 120/230] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:55 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:56.008879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:56.021466 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:56.128434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b7af1-59bc-478c-b085-e12d45319b7f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:56.172696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:56.174530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:56.175024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:56.175407 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:56.176993 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:56.178797 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:56.182671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 111/231] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:56 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:56.188325 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:56.222197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:56.235977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccdc01-e94a-4c61-921a-795356467cf0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:56.275061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:56.275997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:56.276264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:56.276590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/users/94f1d28dd2924962b657e4d9daf4240e/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:56.280433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, user_id=94f1d28dd2924962b657e4d9daf4240e, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:56.280701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:56.283035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:56.304475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 121/232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:56 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/users/94f1d28dd2924962b657e4d9daf4240e/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:56.367240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ca407-5ae7-4d11-b240-6f79c207ccfc None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:56.409469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:56.410358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:56.410644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:56.410922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:56.412166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:56.413778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2397fa-821f-4290-b8b9-f20baf79a773 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:56.416125 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 112/233] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:56 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:56.422166 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:56.605887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa22a5d-f4ab-4328-b33e-9ec03cc25e7b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:56.647557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:56.649463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:56.649948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:56.650417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/users/94f1d28dd2924962b657e4d9daf4240e/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:56.653133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, user_id=94f1d28dd2924962b657e4d9daf4240e, role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:56.654860 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43657f-f6da-43b0-a586-e95644ccad27 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:56.673298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 122/234] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:56 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/users/94f1d28dd2924962b657e4d9daf4240e/roles/afe01609a07c422aa82e18ea2bb45744 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:41:57.655625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5162e-9d6f-4449-95db-51b9b031837e None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:57.656508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5162e-9d6f-4449-95db-51b9b031837e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:57.656508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5162e-9d6f-4449-95db-51b9b031837e None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:57.658060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 113/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:57.670246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:57.671012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:57.671012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:57.694100 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:57.695307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:57.698637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:57.741121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853babc-f7ec-4bca-acb6-a05f37391aed None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:57.741454 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853babc-f7ec-4bca-acb6-a05f37391aed None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:57.741679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853babc-f7ec-4bca-acb6-a05f37391aed None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:57.744752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 114/236] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:57.750602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:57.750862 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:57.751144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:57.780207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:57.781455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:57.835337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 123/237] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:57.873012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:57.917140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 115/238] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:57.944203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.024981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.059830 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e434d55-24e6-4b37-837c-65080b70344a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.120498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74aafc9-5c72-4101-bbf1-fce8643906ed None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.129226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.130427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.130903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.131366 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.132952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:58.134928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:58.136885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 124/239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:41:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:41:58.149365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.163287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.164221 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.165098 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.165383 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.178862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.185405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:58.200060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:58.200995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:41:58.201762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:58.204174 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 116/240] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:58.211112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.394633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618655c-eaf5-4c87-a2c4-a45aabc50a35 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.428142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049dc0b-6e87-45f5-8c18-de9f679e2960 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.452870 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.467121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.467808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.468090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.468365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.469119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:58.480424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:58.502873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.503553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.503847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.512528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.512528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:58.512528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:58.512528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:58.514524 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 125/241] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:41:58 2022] POST /identity/v3/users => generated 293 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:41:58.536078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 117/242] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:58.542658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.673020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:58.700708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77417512-abc8-42cc-8865-a4756262aa4a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.740530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.741653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.742101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.743039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] PATH_INFO: `/v3/users/a84dbfac074d47ac87a8216cf8a4dc21` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:58.773244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] RBAC: Authorizing `identity:get_user(user_id=a84dbfac074d47ac87a8216cf8a4dc21)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:58.779570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae469ce-8bb2-4d32-b75c-789262ad8319 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:58.782253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 126/243] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:58 2022] GET /identity/v3/users/a84dbfac074d47ac87a8216cf8a4dc21 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:58.788259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:58.915529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212fb33-544a-4589-acc6-236ab73d1679 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:58.951723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:58.953145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:58.953629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:58.954084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.017217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.024557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ab04-b557-46dc-9738-f039cf043f98 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.024857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ab04-b557-46dc-9738-f039cf043f98 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.025206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9ab04-b557-46dc-9738-f039cf043f98 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.026531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 127/244] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:59.031897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.032182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.032484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.052748 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:59.054318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:59.064904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.066274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.141490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 128/245] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:41:59.167321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.184991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 118/246] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:41:59.191707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.226229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.258331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.290195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a92ea-f319-43a9-8826-a41fd3550c3d None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.304194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240742b-5e74-42f8-b27d-4346e5a8b4c1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.340515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.341695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.342118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.342531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.355517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6d74fd85-5404-4e34-8f67-25afd2b313a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:59.363883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:59.365126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:41:59.366057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:41:59.374440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.374440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.374440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.374440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.374896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.374976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.375606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 129/247] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:41:59.376812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 119/248] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:59.377468 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c557-4124-4808-97af-8e6e9e46597a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.378008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c557-4124-4808-97af-8e6e9e46597a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.378422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77c557-4124-4808-97af-8e6e9e46597a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.379934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 130/249] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:41:59.383213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.384901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c557-4124-4808-97af-8e6e9e46597a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.501457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31abb1ca-4191-4aa2-a6d8-f3e56713b4f9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.515270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77c557-4124-4808-97af-8e6e9e46597a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.530019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.542765 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.552633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.553657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.554139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.554609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.556913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:59.557432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.559028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ba08-8f4c-4644-8587-79b3cc5b612c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.563564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4410724-519a-4370-947a-3eb941816ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.564230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410724-519a-4370-947a-3eb941816ada None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.564562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410724-519a-4370-947a-3eb941816ada None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.564844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410724-519a-4370-947a-3eb941816ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.565689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4410724-519a-4370-947a-3eb941816ada None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=a84dbfac074d47ac87a8216cf8a4dc21) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:41:59.566005 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4410724-519a-4370-947a-3eb941816ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.568584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4410724-519a-4370-947a-3eb941816ada None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.571245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 120/250] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:59.573510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.574041 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.574515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.575852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 131/251] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/role_assignments?user.id=a84dbfac074d47ac87a8216cf8a4dc21&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:59.581046 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4410724-519a-4370-947a-3eb941816ada None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.615395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:41:59.619593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:41:59.756102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 121/252] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:41:59.761596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4410724-519a-4370-947a-3eb941816ada None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.776348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.802134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.803342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.803342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.803511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] PATH_INFO: `/v3/users/53546c7e89274cea9c4d43b0f97bc8db` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.804935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] RBAC: Authorizing `identity:get_user(user_id=53546c7e89274cea9c4d43b0f97bc8db)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.807088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.809331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 132/253] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/users/53546c7e89274cea9c4d43b0f97bc8db => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:59.814346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.865200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.909535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0be5a1-7c85-43d4-9a89-043ca7f63805 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.934304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626801a1-38b5-4703-bae5-01276fa14606 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:41:59.935144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:41:59.953455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.954609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.955084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.955546 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.969069 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:41:59.974432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:41:59.976201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:41:59.976201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:41:59.976201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:41:59.977495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:41:59.978990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:41:59.979279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:41:59.980111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:41:59.980931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:41:59.981966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 133/254] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:41:59.983566 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 122/255] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:41:59.989272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:41:59.990522 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:00.146452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdfd3-3b7c-436e-b8b6-f793f5d07447 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:00.184389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:00.185084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.185364 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.185640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.186389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:00.186693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:00.188089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823f495-6ded-42a5-b12c-3d8fc72fdef5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:00.197550 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 123/256] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:00.198879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc83c2-61bd-4f14-968a-00dc000e4617 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.199167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc83c2-61bd-4f14-968a-00dc000e4617 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.199440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc83c2-61bd-4f14-968a-00dc000e4617 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.200887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 124/257] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:00.205387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc83c2-61bd-4f14-968a-00dc000e4617 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:00.207475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97af809-edd7-40ba-8984-43edf8449837 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:00.234087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:00.260859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:00.327247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc83c2-61bd-4f14-968a-00dc000e4617 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:00.341140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:00.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.342907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.343374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/users/53546c7e89274cea9c4d43b0f97bc8db/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.348549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, user_id=53546c7e89274cea9c4d43b0f97bc8db, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:00.350282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:00.368478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:00.368684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.368921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.369098 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.370457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:00.372162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d411c-6669-43ae-9b23-2e25ab979563 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:00.374096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 125/258] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:00 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:00.375234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.375590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.375787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.375975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 134/259] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:41:59 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/users/53546c7e89274cea9c4d43b0f97bc8db/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:42:00.382357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:00.416591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:00.417825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:00.620456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 126/260] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:00.644600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:00.651171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126c58-55af-4558-8196-b9bbb24a95b2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:00.698207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:00.741729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:00.743917 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.744549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.745023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.796762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:00.802094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19327bfc-6e6f-4965-ac35-df110c887c35 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:00.837690 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:00.839618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:00.882404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:00.883934 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:00.884280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:00.884779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:00.910391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:00.919839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:00.920776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:00.921542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:00.929508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 127/261] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:00.935833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:00.967841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 135/262] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:42:00.974911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:01.058677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:01.098226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9ff46-8ea7-4e84-b4d2-3181069a951a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.144790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.144790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.144790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.144790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.145849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.147386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.149814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 136/263] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:00 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:01.160685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:01.187362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:01.191507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf907e2-b1c3-407b-91e7-e5ee3592f32e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.257025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.257622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.258480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.258801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.259769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:01.260077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.261515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.277749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 128/264] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:01.283283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:01.351002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7e69f-e183-44ed-bc87-96bed0b31b43 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.418841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.419503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.419783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.420056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.420939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=94f1d28dd2924962b657e4d9daf4240e) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:01.421258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.422721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.429612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 137/265] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:01 2022] GET /identity/v3/role_assignments?user.id=94f1d28dd2924962b657e4d9daf4240e&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:01.487024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00f81-f56f-4c7a-bf8a-12ef5a7074ce None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.542138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.543010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.543292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.543560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] PATH_INFO: `/v3/users/94f1d28dd2924962b657e4d9daf4240e` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.549367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] RBAC: Authorizing `identity:get_user(user_id=94f1d28dd2924962b657e4d9daf4240e)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.550894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.552692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 129/266] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:01 2022] GET /identity/v3/users/94f1d28dd2924962b657e4d9daf4240e => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:01.559245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:01.562968 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:01.734793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154df08d-9eb8-479f-a947-e9f047c49397 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:01.772656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:01.773360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:01.774163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:01.774607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:01.867062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:01.869042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:01.871450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:02.013322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 138/267] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:02.018526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:02.069830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:02.170184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b837b5-b8ac-4fcb-bf0a-b8085065e60c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:02.217451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:02.218326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.218606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.218880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.220083 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:02.221552 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016988ce-79bd-4322-9410-448beb14fcbf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:02.223594 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 130/268] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:02 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:02.229289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:02.414212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c1bb8-3a57-4e2c-affd-3ddfd92ac28f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:02.415925 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:02.443478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811bec-e503-4be9-b7b2-26cee54aafbd None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.443478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811bec-e503-4be9-b7b2-26cee54aafbd None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.443478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811bec-e503-4be9-b7b2-26cee54aafbd None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.443478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 131/269] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:02.450989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25059b66-d0f6-48e3-8fd7-ca7be3ef263a None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.451279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25059b66-d0f6-48e3-8fd7-ca7be3ef263a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.451556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25059b66-d0f6-48e3-8fd7-ca7be3ef263a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.474733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:02.475386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.475666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.475935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.477346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=94f1d28dd2924962b657e4d9daf4240e) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:02.477657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:02.479102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:02.485728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-25059b66-d0f6-48e3-8fd7-ca7be3ef263a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:02.486927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-25059b66-d0f6-48e3-8fd7-ca7be3ef263a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:02.492728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 139/270] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:02 2022] GET /identity/v3/role_assignments?user.id=94f1d28dd2924962b657e4d9daf4240e&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:02.606175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 132/271] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:02.646954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:02.767094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a993f-b3aa-4378-a016-68f051301149 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.767719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a993f-b3aa-4378-a016-68f051301149 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.768246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a993f-b3aa-4378-a016-68f051301149 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.770434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 133/272] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:02.777923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.778706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.779180 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.806460 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:02.808943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:02.810575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0a4e-3651-4cf0-9093-51ad1c8dd2ac None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:02.843925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:02.856075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:02.857370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:02.857881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:02.858295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:02.859721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:02.862469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:02.864302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 140/273] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:02.871226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:02.904449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 134/274] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:02.935819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:02.936778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.037761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057fe628-4022-4411-97af-1d1ab914eb6c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.069071 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1b842-01de-46ac-b3af-6ed20502dc5e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.075790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.077029 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.077594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.078146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.079229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.081490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067ce29-5f6d-4298-9586-dd63ddf9aafd None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.105689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 141/275] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:02 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:42:03.107039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa6418-0dca-4a86-825f-be17e4b11305 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.107325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa6418-0dca-4a86-825f-be17e4b11305 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.107898 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa6418-0dca-4a86-825f-be17e4b11305 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.112445 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 142/276] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:03.115065 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.115355 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.115662 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.119286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.120883 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.121367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.121890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.137708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:03.138563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:03.139020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:03.147520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:03.149512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:03.150289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:03.153020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 135/277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:03.164742 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.196062 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.239435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 143/278] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:03.262511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.290362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.328029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb11-9200-41c9-b4e8-d16da75d2f6b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.366726 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.367665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.368086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.368528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.369442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:03.369891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.371715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.381981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 136/279] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:03.388012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.400024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf59b3a-73e1-43bd-a49d-9380e7e263d6 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.452206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.453534 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.454038 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.454312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.469042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:03.478505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:03.479458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:03.480226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 144/280] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:03.487262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.502297 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.532911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f16241-7603-43bf-8dae-8a8c8eec85ea None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.582323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.583126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.583404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.583673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] PATH_INFO: `/v3/users/55793216a0a349eba52662326a0fa43f` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.606227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] RBAC: Authorizing `identity:get_user(user_id=55793216a0a349eba52662326a0fa43f)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.608718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.609053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.616586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 137/281] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/users/55793216a0a349eba52662326a0fa43f => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:03.619903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.637364 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf550b7a-135e-4eaf-ae27-817f07ae9748 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.686943 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.687635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.687973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.688271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.689009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:03.689320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.690721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104e96a-7181-493e-9f3a-51ac1fa4233a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.708695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 145/282] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:03.712630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93279-4365-4ec5-9ea4-9f1bf3755735 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.712930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93279-4365-4ec5-9ea4-9f1bf3755735 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.713209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93279-4365-4ec5-9ea4-9f1bf3755735 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.715405 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 146/283] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:03.716185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93279-4365-4ec5-9ea4-9f1bf3755735 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.781267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b9fca-5af3-4dfa-aa4a-976659baf509 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.835280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.835876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.836894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.837146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.839168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, user.id=55793216a0a349eba52662326a0fa43f, scope.system=all) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:03.839168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.839717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b930d-c772-428b-a2c4-1942f8779e6a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.845284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 138/284] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/role_assignments?user.id=55793216a0a349eba52662326a0fa43f&scope.system=all&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:03.846564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93279-4365-4ec5-9ea4-9f1bf3755735 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:03.846679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.846860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.847129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.853489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.868850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:03.870033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:03.911873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:03.913604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:03.914306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:03.914900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] PATH_INFO: `/v3/users/a84dbfac074d47ac87a8216cf8a4dc21` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:03.916684 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] RBAC: Authorizing `identity:get_user(user_id=a84dbfac074d47ac87a8216cf8a4dc21)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:03.918389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:03.920508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 147/285] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/users/a84dbfac074d47ac87a8216cf8a4dc21 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:03.926078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:03.941662 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:03.976636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 139/286] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:04.057221 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.078753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51214fdf-9762-4e1c-b710-a8caf0414372 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.126269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.127392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.127788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.128176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.129593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:04.131154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:04.133937 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 148/287] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:03 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:04.139961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.216238 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b2e8e5-c943-42b5-befc-cfb62dad9ec7 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.250415 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:04.264888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.265700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.265981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.266259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.286090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:04.294574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:04.295440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:04.296205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 140/288] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:04.306984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.325688 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:04.355936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cb58b-681f-4eb3-9a3f-7d01a297685f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.417665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.418627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.418910 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.419187 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/users/a84dbfac074d47ac87a8216cf8a4dc21/roles/a677fb6c90ac4ea2ae46d7104319ee06` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.421418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, user_id=a84dbfac074d47ac87a8216cf8a4dc21, role_id=a677fb6c90ac4ea2ae46d7104319ee06)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:04.422999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:04.451346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 149/289] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:04 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/users/a84dbfac074d47ac87a8216cf8a4dc21/roles/a677fb6c90ac4ea2ae46d7104319ee06 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:04.494692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f5f53-242b-44d7-bb6b-85aee17023d9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.535748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.536485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.536778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.537062 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.537977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:04.538292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:04.539981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:04.550120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 141/290] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:04.557360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.712867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38929a-f6a9-408e-9335-ea28a6d0addb None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.756226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.757157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.757436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.757889 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] PATH_INFO: `/v3/users/53546c7e89274cea9c4d43b0f97bc8db` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:04.759292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=53546c7e89274cea9c4d43b0f97bc8db)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:04.760878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:04.762868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 150/291] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:04 2022] GET /identity/v3/users/53546c7e89274cea9c4d43b0f97bc8db => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:04.768016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:04.833084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:04.892618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a354-9e42-4c9f-84d3-fce6cdd3985d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:04.938994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:04.939650 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:04.939938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:04.940217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:05.057773 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:05.059265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:05.125271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:05.174827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 142/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:05.185720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:05.195365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:05.349122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5adf01-c916-4408-b543-06c636126ef4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:05.412288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:05.413377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:05.413667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:05.413976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:05.415782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:05.417268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4adb8a-6302-48af-a939-efb4358f9323 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:05.419066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 151/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:05 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:05.425250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:05.549827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a64db-a016-4931-91c9-182cddf99f5b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:05.588771 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:05.589445 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:05.589728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:05.589997 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:05.590854 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=53546c7e89274cea9c4d43b0f97bc8db) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:05.591167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:05.592660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997985c-9512-494d-81e6-d73f72fc326a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:05.599914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 143/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:05 2022] GET /identity/v3/role_assignments?user.id=53546c7e89274cea9c4d43b0f97bc8db&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:05.868335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e9e0d-ea2c-4427-8bef-7214d2a6b286 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:05.868714 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e9e0d-ea2c-4427-8bef-7214d2a6b286 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:05.868883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e9e0d-ea2c-4427-8bef-7214d2a6b286 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:05.870657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 152/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:05.876113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:05.876419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:05.876722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:05.895228 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:05.896527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:05.914427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:05.985794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 144/296] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:06.015897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e9e0d-ea2c-4427-8bef-7214d2a6b286 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.115908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:06.144070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e9e0d-ea2c-4427-8bef-7214d2a6b286 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.182186 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.182979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.183257 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.183525 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.194609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-3f78793a-8e09-4e17-97ec-63a929e424d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:06.203061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:06.204526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:06.205762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 153/297] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:06.210469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.277244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61bf26-e2d9-406d-a8aa-37548afe181f None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.277885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61bf26-e2d9-406d-a8aa-37548afe181f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.278375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61bf26-e2d9-406d-a8aa-37548afe181f None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.279971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 154/298] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:06.285370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.285912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.286391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.306463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:06.308650 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:06.328690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b5d11-b7c2-40f5-9010-8171c8e5c376 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.371782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.372481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.372767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.373046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.373799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] RBAC: Adding query filter params (name=reader) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:06.374106 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:06.375593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:06.391073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 145/299] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:06.392051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.405758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:06.424452 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 155/300] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:06.442237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.513630 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c37c39-3a48-4eb3-8975-03b25478b55d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.514745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:06.553235 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.554472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.555061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.555472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] PATH_INFO: `/v3/users/5d2cc11a5075482aa10b832154530bbf` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.567619 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e053-0970-46e9-95ec-84f05dfbd143 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.576606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2cc11a5075482aa10b832154530bbf)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:06.578322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:06.585038 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 146/301] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/users/5d2cc11a5075482aa10b832154530bbf => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:06.586889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.608244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.609207 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.609562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.609909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.611342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:06.612988 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:06.620141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 156/302] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:06.621573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:06.671956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:06.711394 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95855bd4-7b80-4a26-955e-c796bb420463 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.745964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d982b-ee12-45b2-8b1d-3ef360d84ac8 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:06.748163 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.748856 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.749140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.749417 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.750779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, user.id=5d2cc11a5075482aa10b832154530bbf, scope.system=all) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:06.750779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:06.752006 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a86a89-a41b-4a4b-94a7-7ac6ce1152cb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:06.763784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 147/303] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/role_assignments?user.id=5d2cc11a5075482aa10b832154530bbf&scope.system=all&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:06.793030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:06.801359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:06.802192 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.802796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.803106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] PATH_INFO: `/v3/groups` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.803903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:06.805972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:06.824338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d85a-ebf4-4359-abb9-7cd5df5eb21d None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.824719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d85a-ebf4-4359-abb9-7cd5df5eb21d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.825203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425d85a-ebf4-4359-abb9-7cd5df5eb21d None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.827296 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 148/304] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:06.828554 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 157/305] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 19:42:06 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:06.835089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:06.835414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:06.835740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:06.857830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:06.859216 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:06.963888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 149/306] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:06.986687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.124428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ecd18-fa17-4f31-9289-65f6e96c7bbc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.173324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.174142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.174428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.174767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.187067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-25d635e4-696a-4e66-8bc2-36801b331866 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:07.195482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:07.196927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:07.197993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 158/307] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:07.202995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.271857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.315112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425740b0-9352-46ac-b13f-870d82e17e3c None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.328809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba7c6-740f-4e88-b801-a70c0d8e247e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.329130 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba7c6-740f-4e88-b801-a70c0d8e247e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.329420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba7c6-740f-4e88-b801-a70c0d8e247e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.331066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 159/308] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:07.337396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.337677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.337953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.356966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.357441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:07.357642 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.357920 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.358187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.358666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:07.358938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:07.359240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:07.360690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:07.374237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 150/309] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:07.380295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.420307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.452782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 160/310] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:07.471718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.496025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6072d-319d-4c98-af8c-29301355701a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.521764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.533503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.534315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.534589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.534860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] PATH_INFO: `/v3/users/55793216a0a349eba52662326a0fa43f` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.536064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] RBAC: Authorizing `identity:get_user(user_id=55793216a0a349eba52662326a0fa43f)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:07.537531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:07.539614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 151/311] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/users/55793216a0a349eba52662326a0fa43f => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:07.545024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.591641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a5bf0-abb3-45fd-bcab-9e261a4ea429 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.631697 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.633514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.634010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.634514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.647280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:07.655537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142fe5-610b-47b8-998a-173c5f45b5d0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:07.656664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:07.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:07.658281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:07.665854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 161/312] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:07.667240 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.693811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.694755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.695614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.695900 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] PATH_INFO: `/v3/system/users/55793216a0a349eba52662326a0fa43f/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.697633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=55793216a0a349eba52662326a0fa43f, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:07.699266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:07.713334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 152/313] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 19:42:07 2022] PUT /identity/v3/system/users/55793216a0a349eba52662326a0fa43f/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:42:07.722218 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.793987 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b790b4c-bfb6-4d2d-a6e7-4fe96c967ed9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:07.835225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:07.836428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:07.836934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:07.837431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:07.838399 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:07.838911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:07.840618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:07.850341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 162/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:07.860713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:07.926155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:07.985095 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652ae8-b1ea-4bef-92be-ea4d22a3f72f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:08.021175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:08.022057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:08.022334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:08.022605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] PATH_INFO: `/v3/users/a84dbfac074d47ac87a8216cf8a4dc21` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:08.028378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] RBAC: Authorizing `identity:get_user(user_id=a84dbfac074d47ac87a8216cf8a4dc21)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:08.028378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:08.028378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 153/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:07 2022] GET /identity/v3/users/a84dbfac074d47ac87a8216cf8a4dc21 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:08.034702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:08.152420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d21f4-d969-4690-8075-19c0971f1106 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:08.160793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:08.190759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:08.191424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:08.191701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:08.191972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:08.266651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:08.268142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:08.375898 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 163/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:08.381338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:08.492042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca665d-7a08-464b-a5bb-6eacff0a4733 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:08.517976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:08.530207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:08.531127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:08.531459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:08.531739 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:08.532981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:08.534426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034699b-6037-4db6-81ac-a88d178ece03 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:08.536432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 154/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:08 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:08.542756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:08.546241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:08.650014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee71502-0ada-4cb7-b3c5-4f66b5b2c5f2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:08.687611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:08.688274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:08.688585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:08.688857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:08.689684 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, scope.project.id=55bf8a308db7471ba78821a3b8026721, user.id=a84dbfac074d47ac87a8216cf8a4dc21) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:08.689993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:08.691424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43df3fe-31a8-4bbc-848e-6aaa18152e1a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 164/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:08 2022] GET /identity/v3/role_assignments?user.id=a84dbfac074d47ac87a8216cf8a4dc21&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:09.262395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9949720-4b32-43d8-b804-de65118b65e6 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.262736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9949720-4b32-43d8-b804-de65118b65e6 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.262924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9949720-4b32-43d8-b804-de65118b65e6 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.264256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 155/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:09.269559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.270027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.270106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.289767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:09.290983 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:09.326770 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505b2f3-7877-44b0-a1a2-a89e91432aba None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.327109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505b2f3-7877-44b0-a1a2-a89e91432aba None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.327332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505b2f3-7877-44b0-a1a2-a89e91432aba None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.328792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 156/320] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:09.334565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.334877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.335124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.353725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:09.354940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:09.363868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:09.371935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 165/321] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:09.394223 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:09.435690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 157/322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:09.459196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:09.511184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359bd00-2b78-47bb-8cd8-6c33136facd4 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:09.553249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:09.553969 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.554392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.554982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.567201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:09.572314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f1de2-3cf8-4ff0-91cb-58746889dade None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:09.575869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:09.576796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:09.577559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:09.582034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 166/323] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:09.582780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:09.599812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:09.615182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:09.615983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.616266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.616572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.617987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:09.619441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:09.621484 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 158/324] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:09.626977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:09.650538 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:09.706341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309818b6-972d-48fa-a6c9-35c8eabcad36 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:09.746131 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:09.746810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.747089 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.747345 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c362249-11a7-4dfd-b484-11b78d8aa5a1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:09.747459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.748104 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:09.748440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:09.749857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:09.760738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 167/325] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:09.766023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:09.792340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:09.793357 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.793677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.793911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] PATH_INFO: `/v3/groups` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.794600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:09.796036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:09.813388 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 159/326] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 19:42:09 2022] POST /identity/v3/groups => generated 226 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:42:09.901042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e90f8c-21b7-4c98-97cd-49902f8d4854 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:09.925129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:09.943477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:09.944385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:09.944949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:09.945849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] PATH_INFO: `/v3/users/5d2cc11a5075482aa10b832154530bbf` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:09.947375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] RBAC: Authorizing `identity:get_user(user_id=5d2cc11a5075482aa10b832154530bbf)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:09.951019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72096019-5a43-4070-b918-f1e9e11c0d5f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:09.951846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 168/327] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:09 2022] GET /identity/v3/users/5d2cc11a5075482aa10b832154530bbf => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:09.957400 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.019987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:10.045750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d8c5-eeb4-4eb4-98cb-ac8ab1315911 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.046037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d8c5-eeb4-4eb4-98cb-ac8ab1315911 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.046321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895d8c5-eeb4-4eb4-98cb-ac8ab1315911 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.047687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 169/328] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:10.053338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.053824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.054143 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.072162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:10.073332 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:10.088371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54090ac-1529-4d64-9de8-f5b1eac18d00 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:10.132139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:10.133602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.134209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.134558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] PATH_INFO: `/v3/system/users/5d2cc11a5075482aa10b832154530bbf/roles/a677fb6c90ac4ea2ae46d7104319ee06` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.136381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d2cc11a5075482aa10b832154530bbf, role_id=a677fb6c90ac4ea2ae46d7104319ee06)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:10.137822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:10.147431 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 160/329] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 19:42:09 2022] PUT /identity/v3/system/users/5d2cc11a5075482aa10b832154530bbf/roles/a677fb6c90ac4ea2ae46d7104319ee06 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:10.172105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 170/330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:10.196724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.340371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa6d15-c96f-4df5-aacc-affbad7f37cb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:10.356443 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:10.383919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:10.385118 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.385575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.386023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.398604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:10.406750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:10.407626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:10.408555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:10.410559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 161/331] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:10.417284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.564636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279dde-17b9-4890-900a-9153ef60afbe None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:10.586102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:10.606592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:10.607185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.607465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.607737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.608527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:10.608837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:10.610238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:10.623474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 171/332] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:10.630135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.785127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da125-6a33-4c59-8617-619854561b2c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:10.836443 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:10.837334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:10.837619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:10.837897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] PATH_INFO: `/v3/users/55793216a0a349eba52662326a0fa43f` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:10.839124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] RBAC: Authorizing `identity:get_user(user_id=55793216a0a349eba52662326a0fa43f)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:10.840613 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73da567c-aac7-4b76-af3d-48bef0283e98 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:10.842576 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 162/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:10 2022] GET /identity/v3/users/55793216a0a349eba52662326a0fa43f => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:10.848616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:10.980628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5ecc-4628-4ca7-9d3d-b8ccfa5f754d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:11.030132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:11.030796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:11.031072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:11.031341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:11.032179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] RBAC: Adding query filter params (role.id=c537ee86dc80416a80e38d5b7769daf6, user.id=55793216a0a349eba52662326a0fa43f, scope.system=all) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:11.032523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:11.033970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddded4-db48-430d-ac95-f668407d83f5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:11.052032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 172/334] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:42:10 2022] GET /identity/v3/role_assignments?user.id=55793216a0a349eba52662326a0fa43f&scope.system=all&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:12.868494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d85ba6-2128-4990-a06b-afc20db19ddf None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:12.869600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d85ba6-2128-4990-a06b-afc20db19ddf None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:12.870349 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d85ba6-2128-4990-a06b-afc20db19ddf None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:12.872648 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 163/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:12.878530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:12.879722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:12.880388 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:12.908147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:12.909851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:12.962852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:12.972657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8a21a-5b51-4fbb-983c-4ed50544912c None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:12.972952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8a21a-5b51-4fbb-983c-4ed50544912c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:12.973233 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8a21a-5b51-4fbb-983c-4ed50544912c None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:12.974624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 164/336] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:12.980468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:12.980760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:12.981037 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.016973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:13.018185 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:13.074574 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 173/337] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:13.092922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:13.119033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:13.166169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 165/338] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:13.207539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:13.281157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb676d-bad7-4390-89d4-3d70fce1a6db None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:13.332731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:13.333705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.333994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.334272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.352438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:13.367154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:13.368149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:13.368942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 174/339] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:13.378863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:13.420869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:13.546127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3b763-b909-47d1-b117-cfd04d88ba10 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:13.573237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203d2f3-e13e-40bc-b656-064491223dc7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:13.631029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:13.633796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.634129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.634406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.663076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:13.665712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:13.668953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.669260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.669551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.670533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:13.670943 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:13.674729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c589e-fb9d-44d4-9df5-d0e7909b5306 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-b1ef3666-5ee4-4de7-a7d9-6f69018089b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:13.685336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:13.686278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:13.687055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 166/340] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:13.688193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3037d4-275f-4eeb-bd2d-8f4cd37e65eb None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.688193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3037d4-275f-4eeb-bd2d-8f4cd37e65eb None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.688458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3037d4-275f-4eeb-bd2d-8f4cd37e65eb None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.694542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 167/341] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:13.695993 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3037d4-275f-4eeb-bd2d-8f4cd37e65eb None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:13.701446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 175/342] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:13.702843 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.703137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.703422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.744857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:13.746341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:13.776840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:13.891052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 176/343] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:13.891909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:13.928510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3037d4-275f-4eeb-bd2d-8f4cd37e65eb None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:13.931233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:13.981860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:13.982575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.982895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.983342 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:13.984256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:13.984755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:13.986305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2c6d0-4a91-4fcc-910f-3d2a5dd99e65 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:13.997145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 168/344] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:13.998744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacba66-a7c7-4783-95e7-a79234a9fa10 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:13.999200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacba66-a7c7-4783-95e7-a79234a9fa10 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:13.999653 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacba66-a7c7-4783-95e7-a79234a9fa10 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.001103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 169/345] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:14.001958 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacba66-a7c7-4783-95e7-a79234a9fa10 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.031785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328c95b-0621-4019-aba6-f3e974a8aeea None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.074668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.075416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.075863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.076063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] PATH_INFO: `/v3/users/5d2cc11a5075482aa10b832154530bbf` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.077673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] RBAC: Authorizing `identity:get_user(user_id=5d2cc11a5075482aa10b832154530bbf)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:14.079382 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:14.080953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 177/346] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:42:13 2022] GET /identity/v3/users/5d2cc11a5075482aa10b832154530bbf => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:14.081805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.116329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.126575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacba66-a7c7-4783-95e7-a79234a9fa10 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.166766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.167816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.168442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.168859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.185672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.188101 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f9c77c96-7cbd-4e5b-8f8d-a20fde3a59be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:14.197996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:14.199195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:14.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:14.200786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e7d01-3466-4782-93cf-2e0576de1465 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.201952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 170/347] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:14.206988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.206988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.206988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.225303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:14.227321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:14.246577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.247248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.247533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.247811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.325480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:14.326467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:14.330517 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 171/348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:14.331670 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.426008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.440893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.449299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 178/349] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:14.450485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.457558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92117ea9-e042-4b6d-9c9c-0105bf191405 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.501931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.502792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.503175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.503557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.504522 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] RBAC: Adding query filter params (role.id=a677fb6c90ac4ea2ae46d7104319ee06, user.id=5d2cc11a5075482aa10b832154530bbf, scope.system=all) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:14.504941 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:14.506493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:14.521919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 172/350] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/role_assignments?user.id=5d2cc11a5075482aa10b832154530bbf&scope.system=all&role.id=a677fb6c90ac4ea2ae46d7104319ee06 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:14.523346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.575546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc1e1-32bf-49c4-9bd0-089c9328f05c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.640764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.641455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.641737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.642015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.642776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:14.643086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:14.644515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:14.658939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 179/351] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:14.659822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.689047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76740f3-0503-4aed-a440-d8f6a2083de6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.740453 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.742053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.742557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.743344 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.749577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.759804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:14.770547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:14.771502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:14.772283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 173/352] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:14.773715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.779282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:14.826400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c2f5-9905-4243-9291-64f7f6e48354 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.870156 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.871044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.871423 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.871704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.873048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:14.874532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30345f5f-6648-4d49-8994-fa1d503c20a3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:14.880443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 180/353] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:14.880443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed638d-208f-4e03-900f-256a1fd59935 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.880443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed638d-208f-4e03-900f-256a1fd59935 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.880443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed638d-208f-4e03-900f-256a1fd59935 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:14.880443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 181/354] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:14.880836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed638d-208f-4e03-900f-256a1fd59935 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:14.932665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a38dca-f80b-409c-a7b5-589433c2578a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:14.982737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:14.983399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:14.983684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:14.983965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.045847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed638d-208f-4e03-900f-256a1fd59935 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.065940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.078086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.079565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953b354b-317a-45b5-9f32-4a141b317861 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.081709 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.094657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.095346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.095629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.095905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.096707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:15.097978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.098417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.109936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 182/355] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:15.110783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.203132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 174/356] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:15.204473 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.204770 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.205051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.223893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:15.225173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:15.243780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30304f2f-1be0-4dcb-9803-c415a04c3f04 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.293253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.294434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.294716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.294989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.304793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.306345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bff863b-c7c4-4f93-a5dc-042ac3be0e2a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.308066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 183/357] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:15.309331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e09c-5135-454e-a49c-188de1aaad07 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.309614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e09c-5135-454e-a49c-188de1aaad07 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.309976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e09c-5135-454e-a49c-188de1aaad07 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.311450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 184/358] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:15.312037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e09c-5135-454e-a49c-188de1aaad07 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.322005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 175/359] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:15.322806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.359669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.378633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.447650 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80717574-a432-4cb7-8a8d-fe3f0aabdf93 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.451176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e09c-5135-454e-a49c-188de1aaad07 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.494127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.494310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.494712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.495195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.495295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.495479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.495637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.495753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.496989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.498452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.500204 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 176/360] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:15.501097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.588630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.590211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.626908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.631348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7c27e-5dec-44a2-9b9a-c9712fdcc2cf None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.656801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.683706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.684458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.684720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.684994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.685833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=757591eb90474b5ba0ad93f33b0a9889) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:15.686181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:15.687612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e58cb-787c-4f71-999c-dd3c3181a057 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:15.693989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 177/361] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:15.695868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.696218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.698904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.727055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:15.728314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:15.753107 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 185/362] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:15.754042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.830495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 178/363] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:15.830495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.889198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fce1b-9601-4302-ac36-b028d1c3d626 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:15.939035 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:15.939863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:15.940145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:15.940508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:15.965267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:15.977141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:15.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:15.978894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:15.984658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 186/364] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:15.984658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:15.984778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.986713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:15.995257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37b6e0-364e-44ae-8557-d7fe13f802ec None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.051389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.052882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.053302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.054200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.056457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.058508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.061077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 179/365] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.061992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.133346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af8936-92d7-41ff-ab48-32c5a8b3ccb7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.189945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.191067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.191355 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.191633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.192911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.194441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.196158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 187/366] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:15 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.197336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.203808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bd5ac-36cd-4d59-9e63-216f5f372cbe None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.249684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.250848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.251323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.251801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.267253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:16.276459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:16.277419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 180/367] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:16.278206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.311676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:16.313312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:16.357407 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690bfe-f633-441b-81b9-da1f0fc88d57 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.407105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.407780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.408063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.408335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.409157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:16.411679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.411679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.420600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 188/368] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.421488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.424279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda64687-4a72-463c-a71a-e1c4d7ed4d53 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.467134 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.467796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.468080 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.468354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.469264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=757591eb90474b5ba0ad93f33b0a9889) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:16.469572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.471031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.479743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 181/369] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.480572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.585510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779eaad-6d91-4448-b878-de0738eccdf6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.592425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:16.612003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb8fca-ff13-4275-b51a-106803ba73de None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.638088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.639080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.639415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.639761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.641205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.642716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.644854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 189/370] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.645771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.649980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:16.658581 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.659275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.659561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.659832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.660640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:16.660933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.662425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.674690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 182/371] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.675974 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.793606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c66e2a-2222-444d-9f05-929895630532 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.798211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54761c-b0dc-409c-84b6-11d2405ffe27 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:16.841769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.842905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.843384 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.847565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:16.848896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:16.848896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:16.849183 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:16.850108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=55bf8a308db7471ba78821a3b8026721) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:16.850564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.852273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.858795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 183/372] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:16.859856 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:16.879019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:16.927214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:16.929542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:16.931892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:17.005435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945748-caa6-4352-8f0e-19d36b89c6ea None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.078645 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.079554 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.080007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.080479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.082919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 190/373] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:17.090941 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.169789 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.171607 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.187406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:17.203118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd477cc-742b-4c9b-bbe7-446380ddbe82 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.205019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:17.261222 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.262114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.262400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.262673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.263990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.265712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.267759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 191/374] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:17.275064 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.303589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 184/375] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:17.309591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.464772 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725dc4f-e75a-4dec-bc66-65963ee3019b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.500276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d6807-096b-471e-9a9f-249b0b9c03f8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.529155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:17.537536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.538963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.539225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.539509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.541519 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.542980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.544654 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 192/376] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:17.551463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.576444 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.577387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.577667 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.577938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] PATH_INFO: `/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.580763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] RBAC: Authorizing `identity:get_project(project_id=160a68f8af7f400ab04b7ad21bdaf62d)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.582209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.584077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 185/377] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:17.589425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.634419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:17.760040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb4ab-e8c6-4d9a-885b-19bfd44019ec None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.771318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdfc3-705f-499b-a9de-122c18e527a8 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:17.802242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.803104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.803383 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.803694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] PATH_INFO: `/v3/groups/8796dfeeafda4fb88e027a1951658e23` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.816896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] RBAC: Authorizing `identity:get_group(group_id=8796dfeeafda4fb88e027a1951658e23)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.818597 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.821356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 186/378] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/groups/8796dfeeafda4fb88e027a1951658e23 => generated 229 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:17.827135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:17.855474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:17.856303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:17.856614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:17.856945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:17.857790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=55bf8a308db7471ba78821a3b8026721) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:17.858109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:17.861665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850c58a-a49f-4298-b7e5-56798f8fe010 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:17.869672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 193/379] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 260 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:17.947652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:18.046335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5df6b-7a2f-4d4e-aa31-abfe8082c98c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:18.100282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:18.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.101882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.102156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.103079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] RBAC: Adding query filter params (group.id=8796dfeeafda4fb88e027a1951658e23, role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=160a68f8af7f400ab04b7ad21bdaf62d) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:18.103394 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:18.104967 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3bef2-a21d-455b-88cb-2f60316d3943 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:18.111808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 187/380] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:17 2022] GET /identity/v3/role_assignments?group.id=8796dfeeafda4fb88e027a1951658e23&scope.project.id=160a68f8af7f400ab04b7ad21bdaf62d&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:18.509754 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924661-c667-4943-9d32-38de511e424b None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.510795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924661-c667-4943-9d32-38de511e424b None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.511311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c924661-c667-4943-9d32-38de511e424b None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.513326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 194/381] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:18.519078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.519658 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.520201 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.544982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:18.546530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:18.643082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 188/382] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:18.679691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c924661-c667-4943-9d32-38de511e424b None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:18.840745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c924661-c667-4943-9d32-38de511e424b None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:18.842252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:18.895381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:18.896189 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:18.896493 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:18.896768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:18.912632 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:18.921209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:18.923010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:18.924362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:18.932711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 195/383] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:18.937584 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.018214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:19.105556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e906f-b966-4de5-8d30-b74b3088e086 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:19.156022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:19.156715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.156991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.157260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.158005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:19.158309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:19.159667 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898a104-5079-4f60-a5da-eee4d7797e47 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:19.168992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 189/384] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:19.177497 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.359529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daec22b-96e0-4579-975d-9d17f7993a31 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:19.398495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edae7-1ea1-4f96-a2de-538526cdd679 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.399139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edae7-1ea1-4f96-a2de-538526cdd679 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.399612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266edae7-1ea1-4f96-a2de-538526cdd679 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.401532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 190/385] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:19.409759 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.410327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.410798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.418425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:19.419541 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.421701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.422182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.425152 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:19.426647 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:19.429134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 196/386] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:19 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:19.434013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:19.434531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.435822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:19.488819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:19.542274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 191/387] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:19.590211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ff7be-dd7c-4914-a49a-b481451534c0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:19.595109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.634625 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:19.635439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.635721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.636175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.637750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:19.639245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:19.641313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 197/388] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:19 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:19.642450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:19.650090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.771885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:19.778230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7aa6f-36fa-4fc5-bf5e-fe5c8a9f9c1c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:19.817034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:19.818225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.818705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.819324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/groups/c446be14b7e34371a95ffb87886a2891/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.821652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, group_id=c446be14b7e34371a95ffb87886a2891, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:19.822407 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b5ab1-7cf4-4611-bdcf-11bba31b95b5 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:19.823512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:19.844316 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 198/389] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:42:19 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/groups/c446be14b7e34371a95ffb87886a2891/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:19.876944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:19.878071 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.878530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.878977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.890975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-0cd122d8-9f0f-4cd4-b8aa-286af3d01fee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:19.900719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:19.901695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:19.902557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:19.905257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 192/390] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:19.914607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:19.958572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ecef-7c70-4d8a-bd6d-7c1d4e623a52 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.959147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ecef-7c70-4d8a-bd6d-7c1d4e623a52 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.959637 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9803ecef-7c70-4d8a-bd6d-7c1d4e623a52 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:19.961506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 193/391] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:19.969237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:19.969763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:19.970187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.001540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:20.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:20.099139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646cf6f-4e02-4e9a-8498-f131dfbb177d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.132507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 194/392] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:20.141792 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.142480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.142833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.143180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.143997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:20.144391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.145861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:20.155812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 199/393] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:20.156750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.166059 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.180775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:20.195963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:20.271323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36d35-3513-4532-aac1-dd338451e996 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.311113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.312008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.312442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.312726 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.337011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:20.351390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd25ddd-fd50-4684-9f96-c65898ae85d3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:20.354618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:20.355472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:20.356690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 195/394] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:20.369056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.433741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.434543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.434819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.435098 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.438993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.440495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:20.442216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 200/395] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:20.448200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.536692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:20.561678 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441df0b2-8796-4e1c-9361-557ac245364d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.602963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.603635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.603915 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.604190 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.604971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:20.605280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.606677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:20.617537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 196/396] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:20.623346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.636259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:20.664236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401039e-0039-4de5-a8f1-f63278879287 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.742569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.743755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.744158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.744848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.747176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.749679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:20.752525 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 201/397] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:20.758173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42233a16-b80b-404f-b983-f2d865c395b5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.758819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.829369 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.830236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.830511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.830780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.832763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.834192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:20.839272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 197/398] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:20.848798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:20.872767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:20.933173 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dce19-9a23-4c91-a4c4-f9910188700b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:20.982873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:20.983766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:20.984203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:20.984501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889/groups/c446be14b7e34371a95ffb87886a2891/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:20.987068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] RBAC: Authorizing `identity:create_grant(project_id=757591eb90474b5ba0ad93f33b0a9889, group_id=c446be14b7e34371a95ffb87886a2891, role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:20.989098 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2123821-8101-4ca4-831b-3666fb0e0eba None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:21.007493 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 202/399] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:42:20 2022] PUT /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889/groups/c446be14b7e34371a95ffb87886a2891/roles/afe01609a07c422aa82e18ea2bb45744 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:21.009863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6f8a-a491-4bf8-94e0-93d1639b2279 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.010206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6f8a-a491-4bf8-94e0-93d1639b2279 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.010494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d6f8a-a491-4bf8-94e0-93d1639b2279 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.012268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 203/400] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:21.018807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd783-0bf5-4af9-a80c-06f799782881 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.019087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd783-0bf5-4af9-a80c-06f799782881 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.019361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fd783-0bf5-4af9-a80c-06f799782881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.043200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ca2fd783-0bf5-4af9-a80c-06f799782881 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:21.045632 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ca2fd783-0bf5-4af9-a80c-06f799782881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:21.070906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:21.081508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0140-53db-4b9a-9a81-9ea7aafe9f5e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.143971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.144943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.145224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.145513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.146941 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:21.149144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703ffd7-0c56-44d0-87f3-f9407f539ee3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:21.155941 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 198/401] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:20 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:21.157586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6345-71c0-4dfb-ad8e-dd82d5b61fad None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.157945 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6345-71c0-4dfb-ad8e-dd82d5b61fad None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.158287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e6345-71c0-4dfb-ad8e-dd82d5b61fad None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.164532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 199/402] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:21.164532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e6345-71c0-4dfb-ad8e-dd82d5b61fad None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.173920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 204/403] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:21.176738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.177280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.177703 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.213494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:21.215467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:21.292932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:21.310549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 205/404] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:21.311915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.339733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e6345-71c0-4dfb-ad8e-dd82d5b61fad None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.378961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.379965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.380250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.380561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/groups/c446be14b7e34371a95ffb87886a2891/roles/c537ee86dc80416a80e38d5b7769daf6` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.383394 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, group_id=c446be14b7e34371a95ffb87886a2891, role_id=c537ee86dc80416a80e38d5b7769daf6)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:21.384991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:21.401698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 200/405] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:42:21 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/groups/c446be14b7e34371a95ffb87886a2891/roles/c537ee86dc80416a80e38d5b7769daf6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:21.408501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.411255 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:21.452009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d21774-8d0d-461b-9d2e-fd53a8eb1dec None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.525253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.526088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.526370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.526645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.539589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb51fb3-6df6-4946-9701-79ab4ef984f5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.569795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:21.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:21.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:21.579559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 206/406] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:21.587803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.588710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.589061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.589340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.589453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.609261 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:21.614937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:21.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:21.633907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:21.634673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:21.640754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:21.645455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.765565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdf134-edf2-4a1b-be7f-3825858b4717 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.813994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691141-6144-418d-a660-ba4bea467333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.814652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ac81e-c766-4800-ba20-c8632b4b7d6f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:21.814762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691141-6144-418d-a660-ba4bea467333 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.814936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691141-6144-418d-a660-ba4bea467333 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.815203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691141-6144-418d-a660-ba4bea467333 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.815952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52691141-6144-418d-a660-ba4bea467333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:21.816256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52691141-6144-418d-a660-ba4bea467333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:21.817955 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52691141-6144-418d-a660-ba4bea467333 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:21.821757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:21.832661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 207/408] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:21.840563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691141-6144-418d-a660-ba4bea467333 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:21.885566 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:21.886247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:21.886525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:21.886799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:21.887667 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:21.887979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:21.889446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:21.915129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 202/409] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:21.916632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.039926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:22.084831 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c04d73-4b9e-4afe-a67b-713cd2ebe806 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.102573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691141-6144-418d-a660-ba4bea467333 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.121104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.121968 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.122244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.122512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.123715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.125172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.127076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 203/410] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:22.133049 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.186618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.187514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.187798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.188075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] PATH_INFO: `/v3/groups/8796dfeeafda4fb88e027a1951658e23` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.189487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] RBAC: Authorizing `identity:get_group(group_id=8796dfeeafda4fb88e027a1951658e23)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.190945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.192998 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 208/411] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:21 2022] GET /identity/v3/groups/8796dfeeafda4fb88e027a1951658e23 => generated 229 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:22.205868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.265115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4df8b7-1482-441c-ad5f-4838f862ddbd None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.294813 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:22.304981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.305847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.306231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.306512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.307761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.309402 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.311535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 204/412] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:22 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:22.317305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.388817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:22.449615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2badb-9268-4afa-afe2-a3a807e004c0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.478691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e4885-3f9e-4b80-b81f-9157bec3c3bd None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.502570 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.503544 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.503891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.504167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721/groups/c446be14b7e34371a95ffb87886a2891/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.509159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=55bf8a308db7471ba78821a3b8026721, group_id=c446be14b7e34371a95ffb87886a2891, role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.510601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.536534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 205/413] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:42:22 2022] PUT /identity/v3/projects/55bf8a308db7471ba78821a3b8026721/groups/c446be14b7e34371a95ffb87886a2891/roles/afe01609a07c422aa82e18ea2bb45744 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:22.553695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.554628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.554929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.555238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] PATH_INFO: `/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.557162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] RBAC: Authorizing `identity:get_project(project_id=160a68f8af7f400ab04b7ad21bdaf62d)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.558675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245dcdf3-d413-4e85-a9ff-a593b7abd64d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.560808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 209/414] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:22 2022] GET /identity/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d => generated 344 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:22.566444 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.637018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6691-5900-44ef-a55e-2edecd19998c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.637339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6691-5900-44ef-a55e-2edecd19998c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.637707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6691-5900-44ef-a55e-2edecd19998c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.639454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 210/415] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:22.646118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.646434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.646666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.659158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:22.681692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:22.683214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:22.723855 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee62d1d-fead-4803-93f8-30f4d27610f6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.797311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 211/416] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:22.801393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.802545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.803020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.804043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] PATH_INFO: `/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d/groups/8796dfeeafda4fb88e027a1951658e23/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:22.806345 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] RBAC: Authorizing `identity:create_grant(project_id=160a68f8af7f400ab04b7ad21bdaf62d, group_id=8796dfeeafda4fb88e027a1951658e23, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:22.807976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:22.823402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:22.828605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 206/417] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:42:22 2022] PUT /identity/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d/groups/8796dfeeafda4fb88e027a1951658e23/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:22.902813 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:22.951269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f1b6a-8324-4c0c-8882-3042eeaba9c6 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:22.997158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:22.997989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:22.998270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:22.998544 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.018420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:23.027293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:23.028514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:23.029649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:23.032871 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 212/418] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:23.039848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.156336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d73f8-5fc3-4a45-a854-43262e793155 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:23.181751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:23.194561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:23.195483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.196047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.196579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.197661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] RBAC: Adding query filter params (name=member) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:23.197969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:23.199367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:23.209917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 207/419] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:23.215376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.359459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7c82d-cbf8-4d5d-96e5-f87e61ca2c2f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:23.387341 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:23.399661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:23.400313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.400617 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.400891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.502601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:23.504049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a67807-53d7-439f-80f3-6fcde6bb2ac3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:23.644218 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 213/420] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:42:23.649867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.675321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d651a0-d718-4cfb-ab9a-551c9758de1a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.675715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d651a0-d718-4cfb-ab9a-551c9758de1a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.676155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d651a0-d718-4cfb-ab9a-551c9758de1a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.679881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 214/421] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:23.684287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.684636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.684918 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.726432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:23.727677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:23.761771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:23.776561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15644f2-6487-48dc-8da1-2fef82cd7bff None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:23.812198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:23.813212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.813639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.813919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.815168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:23.816638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:23.819163 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 208/422] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:23.827679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.885831 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:23.904829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 215/423] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:23.929258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:23.945694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1b8a8-dc35-4d86-b6a2-1a74023306d4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:23.988541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:23.989406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:23.989685 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:23.989956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:23.991844 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:23.993319 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:23.995518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 209/424] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:24.001885 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.091861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0d533-a19b-4f9c-93b8-a8fa77f4cdc5 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.130933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c488a3-1d18-4326-a955-b744415fd878 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.153217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:24.176506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.177624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.177976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.178351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.179250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=757591eb90474b5ba0ad93f33b0a9889) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:24.179627 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:24.181309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:24.188860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 210/425] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:24.218418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.219254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.219534 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.219806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.253640 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e2d0d23c-09e0-478e-81c8-ae7cb6a63c8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:24.262674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:24.263768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:24.264797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:24.268504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 216/426] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:24.270086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.321494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f91f-72ad-4b75-86a7-945fa4f75eec None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.321841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f91f-72ad-4b75-86a7-945fa4f75eec None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.322037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729f91f-72ad-4b75-86a7-945fa4f75eec None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.323769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 217/427] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:42:24.330284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.330665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.331016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.369793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:24.370930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:24.395828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:24.435111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a97d-1b2c-499f-a9e7-c8d0fdb837d3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.473321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.473992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.474273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.474545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.476014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:24.476323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:24.477785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:24.491028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 211/428] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:24.492251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.502689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 218/429] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:24.528977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.539103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:24.606799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ec2e3-7a52-4768-b835-2451468c4769 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.650148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.650860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.651189 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.651467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.653893 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dc4c1-38bd-414e-926c-795bda2a34ef None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.693263 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.694301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.694786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.695160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.706481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:24.717055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:24.718154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:42:24.719103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:24.724433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 219/430] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:24.729822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.736824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:24.748806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:24.750970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:24.808879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:24.865823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cc9b-ba25-4564-a27f-725517372cf7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:24.889689 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 212/431] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:24.896376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:24.915067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:24.916477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:24.917033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:24.917517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:24.918500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:24.919018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:24.920711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:24.932020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 220/432] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:24.937441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.009879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e611b83-3c92-4dfa-be0e-666d2383284e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.017508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.050631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.050631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.050631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.050631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] PATH_INFO: `/v3/projects/757591eb90474b5ba0ad93f33b0a9889` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.051399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] RBAC: Authorizing `identity:get_project(project_id=757591eb90474b5ba0ad93f33b0a9889)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.052880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.060436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 213/433] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/projects/757591eb90474b5ba0ad93f33b0a9889 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:25.062494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.066740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea1610-f6fd-4b88-87c7-69d64a342ef4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.079967 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.121017 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.121715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.122001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.122264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.227232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.228831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.232139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5705ec-d7e9-4f66-8569-1be327c10603 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.282052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.282935 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.283214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.283488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.285307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.287048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.290587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 214/434] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:25.296966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.311061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.359721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 221/435] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:25.364817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.404928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.417170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf21fd-deb4-47df-99d2-aea8f8dd6331 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.456757 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.457413 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.457688 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.457976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.459976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=757591eb90474b5ba0ad93f33b0a9889) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:25.459976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.461424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e55467-4b44-46cb-bc72-fe9e555565d9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.468311 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 215/436] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=757591eb90474b5ba0ad93f33b0a9889&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:25.469439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce2d0f-6983-4c4a-8ebc-7cb390f3b8b3 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.469784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce2d0f-6983-4c4a-8ebc-7cb390f3b8b3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.470046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce2d0f-6983-4c4a-8ebc-7cb390f3b8b3 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.471353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 216/437] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:25.480620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.480620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.480620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.502530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:25.504115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:25.506997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c6a56-5f3c-432b-ba95-a4e44a260922 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.555607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.556600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.556885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.557229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.558710 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.560584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.567789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 222/438] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:25.570416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.591507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 217/439] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:25.615671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.617885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.703032 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5beea5-1fb2-4f10-94f4-43446797f0cc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.715926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.743588 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58923b5-768f-444a-a93c-58709662611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.744300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58923b5-768f-444a-a93c-58709662611a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.744883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58923b5-768f-444a-a93c-58709662611a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.745144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58923b5-768f-444a-a93c-58709662611a None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.746971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58923b5-768f-444a-a93c-58709662611a None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.748599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58923b5-768f-444a-a93c-58709662611a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.750672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 223/440] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:25.751491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb3ecf-720e-4634-9723-a1cf6dc46079 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.759290 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58923b5-768f-444a-a93c-58709662611a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.799893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.800937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.801291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.801576 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.820857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:25.830486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:25.831468 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:25.832234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 218/441] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:25.838430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:25.917336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58923b5-768f-444a-a93c-58709662611a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:25.917336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:25.975907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:25.977689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:25.977689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:25.977689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:25.978660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=c537ee86dc80416a80e38d5b7769daf6, scope.project.id=55bf8a308db7471ba78821a3b8026721) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:25.979431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:25.983306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888126e9-9f48-4eed-95c2-58c7008c5751 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:25.987963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 224/442] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=c537ee86dc80416a80e38d5b7769daf6 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:26.008864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ae0e6-a583-4f84-8881-22b290afb11b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.013361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f52a7-82df-44ef-81fd-489ad0c7c33f None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.013361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f52a7-82df-44ef-81fd-489ad0c7c33f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.013361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f52a7-82df-44ef-81fd-489ad0c7c33f None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.013361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 225/443] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:26.019840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.019840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.020160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.044507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:26.045728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:26.058554 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.066499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.067393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.067698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.067977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.069230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:26.069552 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.071007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.081419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 219/444] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:26.087332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.138189 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 226/445] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:26.161548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.224247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b039b-0670-4c39-857f-317ecb8e6ef5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.271247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.271890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.272687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.272687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.291254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.298053 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e2328-73b9-46f7-86ee-7ab2efd5cd89 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.348580 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.348926 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.349212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.349486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.354220 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.364064 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:26.368447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.368705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:26.373389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:26.374631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:26.375735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 227/446] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:26.380228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.484038 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 220/447] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:42:26.489513 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd899ade-8ed4-42dd-ac8f-1224032b6606 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.489793 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.529115 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.529968 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.530702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.530831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.531857 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:26.532301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.534768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.545471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 228/448] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:26.550980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.571520 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.593094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.604521 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e01d1-5c96-45c2-8ee1-a4fc0cc68279 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.641412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.642287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.642562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.642834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] PATH_INFO: `/v3/projects/55bf8a308db7471ba78821a3b8026721` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.644048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] RBAC: Authorizing `identity:get_project(project_id=55bf8a308db7471ba78821a3b8026721)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.645528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 221/449] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/projects/55bf8a308db7471ba78821a3b8026721 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:26.653411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.675635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de8a62-0104-4ca0-867b-d32bc1b3bd1a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.710956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.711609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.711888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.712180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.765575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d687d4f-38ab-4424-a329-e737997f1664 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.784971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.786390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.801298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.802151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.802423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.802892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] PATH_INFO: `/v3/groups/c446be14b7e34371a95ffb87886a2891` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.804084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] RBAC: Authorizing `identity:get_group(group_id=c446be14b7e34371a95ffb87886a2891)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.805547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.807048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.807156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 222/450] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/groups/c446be14b7e34371a95ffb87886a2891 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:26.812797 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.841113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:26.896042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 229/451] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:42:26.903374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:26.917449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffbb37-3271-4f7f-93db-15a5a62e57d1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:26.953690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:26.954355 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:26.954631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:26.954907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:26.955738 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] RBAC: Adding query filter params (group.id=c446be14b7e34371a95ffb87886a2891, role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=55bf8a308db7471ba78821a3b8026721) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:26.956056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:26.957543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:26.976431 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 223/452] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/role_assignments?group.id=c446be14b7e34371a95ffb87886a2891&scope.project.id=55bf8a308db7471ba78821a3b8026721&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:27.019747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff64098-0dd7-4843-9d9e-ee93f1cfcb06 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:27.040712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:27.060192 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:27.061108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:27.061388 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:27.061658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] PATH_INFO: `/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:27.062939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] RBAC: Authorizing `identity:get_project(project_id=160a68f8af7f400ab04b7ad21bdaf62d)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:27.068518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:27.068518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 230/453] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:42:26 2022] GET /identity/v3/projects/160a68f8af7f400ab04b7ad21bdaf62d => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:27.072301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:27.188059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:27.196700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b94d8b-e944-45b4-837a-682bcd9917f8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:27.235033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:27.235896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:27.236178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:27.236487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] PATH_INFO: `/v3/groups/8796dfeeafda4fb88e027a1951658e23` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:27.237672 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] RBAC: Authorizing `identity:get_group(group_id=8796dfeeafda4fb88e027a1951658e23)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:27.239107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5bb2d-bc0f-4b53-bebc-6b9abb56fe5e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:27.240830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 224/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:42:27 2022] GET /identity/v3/groups/8796dfeeafda4fb88e027a1951658e23 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:27.247211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:27.368690 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507000-e4ec-46dd-8939-2d627396e374 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:27.404024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:27.404708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:27.404990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:27.405264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:27.406084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] RBAC: Adding query filter params (group.id=8796dfeeafda4fb88e027a1951658e23, role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=160a68f8af7f400ab04b7ad21bdaf62d) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:27.406397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:27.407834 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec0d12-6b68-4326-ad37-0a7d68f09321 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:27.414893 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 231/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:42:27 2022] GET /identity/v3/role_assignments?group.id=8796dfeeafda4fb88e027a1951658e23&scope.project.id=160a68f8af7f400ab04b7ad21bdaf62d&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:30.462863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9484e-d323-4ee7-a901-c0effa0f9878 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.464732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9484e-d323-4ee7-a901-c0effa0f9878 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.465723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9484e-d323-4ee7-a901-c0effa0f9878 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.469324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 225/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:30.475163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.475966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.475966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.501784 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:30.503015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:30.567592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:30.603868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 232/457] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:30.634214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a8372-717d-4047-8ba6-6834fd22edf3 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.635277 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a8372-717d-4047-8ba6-6834fd22edf3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.636176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a8372-717d-4047-8ba6-6834fd22edf3 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.640000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 226/458] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:30.644446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:30.647992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb363d-bf8b-40eb-b376-4640c42c8788 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.648966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb363d-bf8b-40eb-b376-4640c42c8788 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.649863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb363d-bf8b-40eb-b376-4640c42c8788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.681361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-09cb363d-bf8b-40eb-b376-4640c42c8788 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:30.682931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-09cb363d-bf8b-40eb-b376-4640c42c8788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:30.763102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 227/459] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:30.764818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25288e6-752e-4635-a43f-ac247b1302a3 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.765257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25288e6-752e-4635-a43f-ac247b1302a3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.765663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25288e6-752e-4635-a43f-ac247b1302a3 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.767143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 228/460] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:30.781897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.782434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.782858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.783419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c505c-d119-4f9b-9be4-d768c19ef659 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:30.808835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:30.810046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:30.825136 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:30.826410 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:30.826872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:30.827416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:30.839464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:30.841429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:30.866181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:30.867095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:30.867976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:30.867976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:30.867976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:30.867976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:30.870655 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 233/461] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:30.871780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:30.887220 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 229/462] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:30.888284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:30.910499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:30.917752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:31.024720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989362b6-81ab-48b4-b52e-c0a84fbd281e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.028760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4acbb5-a11e-4ea5-9026-9c542d635b4e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.062670 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.063617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.064139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.064618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.065500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:31.065955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.067485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.081786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.082575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.083024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.083441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.088210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 230/463] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:31.089284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.101835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.103447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:31.129396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:31.130542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:31.132284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:31.132284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:31.132284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:31.132284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:31.132284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 234/464] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:31.132617 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.186290 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:31.208030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf20a1-d3c1-4022-b85a-ce881fed1fcf None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.248789 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.250054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.250488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.250847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:31.251059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.262587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.264600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.271300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a962163-a4f8-4139-a480-f5ce89184544 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:31.285781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:31.287341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:31.288207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:31.288207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:31.288207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:31.288207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:31.290052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 231/465] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:31.291028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.320878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.322273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.322697 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.323246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.324847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.327942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.354165 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 235/466] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:31 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:31.355982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.434019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c2e1c-8d03-4c68-9080-95d26f9ba467 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.486226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:31.495453 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.497117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.497861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.498661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.500320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:31.501200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.503865 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.517907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 232/467] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:31.528457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdf1a6-2e95-4c88-a89d-abf619b14dcb None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.530348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.612358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.614824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.617862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.618420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.619650 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:31.620457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.622938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.644376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:31.654037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 236/468] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:31.661413 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:31.742702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e27398-54f2-42e6-ad6a-acfa0b039d5c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.786190 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.787585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.788354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.789038 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.790181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.792068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe6a2f-5d26-4a13-a807-70414004696c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.816240 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64ed7-ac34-4d2d-8ee6-ce4d6ee6b356 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:31.822210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 233/469] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:31 2022] POST /identity/v3/users => generated 248 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:31.875925 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:31.876618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.876904 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.877180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.877910 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:31.879494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defaba16-c206-4780-9631-f5c8b23b0b10 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:31.914939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 237/470] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:31 2022] POST /identity/v3/users => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:31.945992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a323-2de0-4255-9c2f-c4d07970a434 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.946361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a323-2de0-4255-9c2f-c4d07970a434 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.946710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78a323-2de0-4255-9c2f-c4d07970a434 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.948720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 234/471] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:31.954495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:31.954787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:31.955065 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:31.986482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:31.988320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:32.017297 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c183d-91cc-4105-8142-b9d832425a2c None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.017517 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c183d-91cc-4105-8142-b9d832425a2c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.017720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c183d-91cc-4105-8142-b9d832425a2c None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.019353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 235/472] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:32.027152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.027599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.027994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.060423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:32.061710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:32.085699 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.114661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 238/473] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:32.156590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.157836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.188959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 236/474] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:32.214905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.318566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb7084-29e2-4aff-b1a8-1486ac529070 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:32.342424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7287fea-e26f-4f31-b64a-9a90bcf90f84 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:32.371887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:32.372914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.373294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.373667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.387365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:32.389335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:32.405249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:32.406199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.406681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.407106 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.408066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:32.409744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:32.410979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:32.412949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:32.413795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:32.413795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:32.413795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:32.413795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:32.419159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 239/475] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:32.420557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.430910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.sql.core [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:42:32.430910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:42:32.430910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [parameters: {'id': 'af7ea5ad95c84426bcbf43f5c2e07d8c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:42:32.430910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86704) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application conn.query(q) May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result.read() May 19 19:42:32.457237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return ref.to_dict() May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:42:32.458139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application next(self.gen) May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application yield resource May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application next(self.gen) May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise self.value May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application session.commit() May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:42:32.469226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.session.flush() May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application compat.raise_( May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:42:32.470795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:42:32.473275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application conn.query(q) May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result.read() May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:42:32.475354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [parameters: {'id': 'af7ea5ad95c84426bcbf43f5c2e07d8c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:32.477162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:42:32.479104 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.480935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.481064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 237/476] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:42:32 2022] POST /identity/v3/roles => generated 144 bytes in 258 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:32.483778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.574326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.635494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d048133-ca17-4103-86ea-d61604cd61c9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:32.651199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc52c6f-f23e-4699-8374-589881fec4c5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:32.699177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:32.700337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.700823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.701274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.713599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:32.722545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:32.723497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:42:32.725308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:32.728329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 238/477] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:42:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:32.733127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:32.733768 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.733873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.734072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.734342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.735072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:32.735465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:32.736879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:32.758311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 240/478] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:32.766144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:32.850684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac928c-4061-44f2-a4d4-e2112ba12b04 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:32.853255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.891020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:32.892109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:32.892639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:32.893117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:32.894076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:32.894584 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:32.896333 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ae0f8-c108-4755-b83b-77bd473329c4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:32.906480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 239/479] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:32.958031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:32.972905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf68e76-2d27-4050-b875-025d51e52719 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:33.038779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:33.039830 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:33.040317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:33.040815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:33.041753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:33.043439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2eaa5d-a246-42a8-8106-e8fa643346cf None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:33.082767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 241/480] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:32 2022] POST /identity/v3/users => generated 249 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:34.684735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560a966-60c8-439c-a098-ee446a54d16d None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.685248 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560a966-60c8-439c-a098-ee446a54d16d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.685447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560a966-60c8-439c-a098-ee446a54d16d None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.686955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 240/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:34.702559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.703195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.703195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.729403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:34.730604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:34.795388 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25a879-ac90-466e-a4a9-6a04d8160f22 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.795727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25a879-ac90-466e-a4a9-6a04d8160f22 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.795945 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25a879-ac90-466e-a4a9-6a04d8160f22 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.797828 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 241/482] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:34.803893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:34.804151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:34.804467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:34.824384 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:34.826449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:34.845751 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 242/483] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:34.889408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:34.939771 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 242/484] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:34.994696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.051343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.073825 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf89104-f67b-4b4f-a5ae-53ac0a09d8ed None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.126260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.127085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.127365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.127635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.139837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.142077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:35.149715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba828ca2-b16a-4375-8c03-0c8b058fc667 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-50e39c06-80f8-41c1-85da-48c926413c31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:35.152322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:35.153380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:35.156476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 243/485] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:35.157891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d1a4-3ea9-4820-a4dc-85464c6601b2 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.158328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d1a4-3ea9-4820-a4dc-85464c6601b2 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.158749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d71d1a4-3ea9-4820-a4dc-85464c6601b2 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.160197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 244/486] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:35.165008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71d1a4-3ea9-4820-a4dc-85464c6601b2 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.204055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.205354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.205779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.206598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.220788 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-336830b2-fc27-4d37-9061-e3e2cf47e2b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:35.232498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:35.233455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.234253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:35.237189 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 243/487] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:35.238547 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.238994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.240207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.263366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:35.264628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:35.309628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d71d1a4-3ea9-4820-a4dc-85464c6601b2 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.341980 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 244/488] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:35.349431 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.357598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.358463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.358872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.359309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.360773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:35.361217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:35.362716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:35.373459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 245/489] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:35.379277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.392338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.483669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.504829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d177883-3b3e-40b2-a546-a2db6e5074ce None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.509111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26635698-777e-43f1-89cb-6f7cc20b5859 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.549259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.550103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.550536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.550904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.551790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:35.552497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:35.553792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:35.554140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.555096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.555849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.556286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.563930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 245/490] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:35.564969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.571778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:35.573490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:35.600147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:35.601265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:35.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:35.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:35.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:35.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:35.605234 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 246/491] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:35.606685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.695541 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.709942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2107db-4670-4213-936e-987e031355e3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.748537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.749438 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.749822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.750203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.755973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a3c7d-9927-4b1d-b188-1e1a3c5d089c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.762810 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:35.770916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:35.772271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:35.773299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:35.774850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 246/492] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:35.780440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.791022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:35.800153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.801123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.801321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.801552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.814001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:35.815458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:35.834815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:35.835728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 247/493] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:35.838231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.914471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d0588a-03f1-47c3-a923-8e79441a05cd None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:35.953112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:35.953779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.954058 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.954332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.955073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:35.955383 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:35.956902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5c16e-fef6-4751-bb49-21f74efc6696 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 247/494] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be3ce-85f3-4f74-b9dc-f03bb76dcf0e None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be3ce-85f3-4f74-b9dc-f03bb76dcf0e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70be3ce-85f3-4f74-b9dc-f03bb76dcf0e None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 248/495] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:35.979981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be3ce-85f3-4f74-b9dc-f03bb76dcf0e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:35.992766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ba1b1-bc85-4d5a-925c-19623eddb765 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.004730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.033584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.034256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.034537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.034814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.035555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:36.035888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.037793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.047412 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 248/496] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:36.048084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.055622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.096503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70be3ce-85f3-4f74-b9dc-f03bb76dcf0e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.137090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.137751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.138031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.138305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.139032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:36.139335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.140741 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24aa98-4658-42e3-b5a4-020d400918ea None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.152886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 249/497] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:36.154103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf16a-a1c1-46f6-ab09-48d67a5ba9d5 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.154494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf16a-a1c1-46f6-ab09-48d67a5ba9d5 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.154663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf16a-a1c1-46f6-ab09-48d67a5ba9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.166141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd75361-2ab5-4e80-9b0b-851ab8278f1a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.175487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-4bdbf16a-a1c1-46f6-ab09-48d67a5ba9d5 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:36.176760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-4bdbf16a-a1c1-46f6-ab09-48d67a5ba9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:36.201275 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.202073 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.202360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.202635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.218803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.220476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:36.236810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:36.237932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 249/498] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:36.239030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.251508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 250/499] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:36.252947 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57039-e9a6-439b-a798-d0f390c31e5b None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.253236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57039-e9a6-439b-a798-d0f390c31e5b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.253508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57039-e9a6-439b-a798-d0f390c31e5b None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.255008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 251/500] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:36.256218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57039-e9a6-439b-a798-d0f390c31e5b None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.273686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.325837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.348829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b666a2f-1aef-4732-83ee-e68e69170d5c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.373049 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57039-e9a6-439b-a798-d0f390c31e5b None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.385485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.386288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.386566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.386838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.398029 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.399527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.409707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.410499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.410771 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.411041 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:36.417286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:36.418360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:36.419401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:36.419401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:36.419401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:36.419401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:36.419660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.421045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.424484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 250/501] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:36.425389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-64af9b6b-b1f0-4e7b-8916-cb8a187c1dc2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:36.438151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:36.439378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:36.440487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 252/502] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:36.441167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.441244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.441498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.459587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:36.460893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:36.519470 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.537670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bea04-5460-4154-a783-e24c0c4b9513 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.542978 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 253/503] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:36.550150 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.572859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.573528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.573822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.574095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.575736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:36.576189 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.578419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.599225 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.620306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 251/504] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:36.621190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.662596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13cff-c577-49d6-8403-4039ec1519a8 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.703026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.703811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.704095 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.704371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.715405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.716892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.727989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796864-127e-4fcd-b536-93d47bba771f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:36.734693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:36.735761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:36.736732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:36.736732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:36.736732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:36.736732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:36.736732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 254/505] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:36.741568 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.772682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.773260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.773643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.773914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.774627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:36.774932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.788213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.802622 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 252/506] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:36.803616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.806792 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.847106 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:36.851197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f819a2-a779-470e-a8fb-5c42f61ea727 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.892715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.893398 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.893677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.893949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.894803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:36.895110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:36.896954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:36.922045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac907d-cf8d-40ee-89dd-45a99fc13492 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:36.934119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 255/507] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:36.934955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:36.959490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:36.960287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:36.960598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:36.960875 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:36.972170 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:36.980099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:36.981083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:36.981872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 253/508] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:36.982871 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.045427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.050207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e967f76-21d3-48e8-83da-f28b4fc03cf7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.086761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.087259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b838a-647a-4a8b-b8a3-31dc8b146226 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.087404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.087681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.087951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.117206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.129999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.131249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.131535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.131805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.132570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:37.132878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.134295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.146141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 254/509] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:37.147037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.164672 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.166277 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.252893 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bfd64-9d4d-489f-be80-bf8e8acb66e2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.277143 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.277340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 256/510] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:37.278009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.290853 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.291735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.292018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.292289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.299658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.301076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:37.316758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:37.318097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 255/511] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:37.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.341347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.388068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66276e-acf3-4d59-ae6c-6343ab7d37fb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.424192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.424859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.424965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98438a2b-1135-4f9d-acd6-35ef5aee038e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.425142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.425411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.460415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.461080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.461358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.461629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.462404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:37.462712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.464160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.473206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 256/512] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:37.474639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.495507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.497005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.524654 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.556720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.584154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdcde4-7733-4978-8352-b80a570a261b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.605134 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 257/513] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:37.606372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.637007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.637812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.638162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.638504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.639281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.640861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.661567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 257/514] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:37 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:42:37.672519 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.735208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b584c15-5706-4b49-a3de-0f6f17d6f756 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.775785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.776637 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.776924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.777203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.789790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.791271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.793070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.798492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b00a83-c6d1-4808-ad3b-90d6cc2ac6b6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:37.811316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:37.812393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:37.813322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:37.813322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:37.813322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:37.813322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:37.813322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 258/515] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:37.813827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.827545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:37.855942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.857031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.857031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.857243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:37.858039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:37.858814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:37.859791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:37.879921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 258/516] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:37.881947 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:37.943167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbcef6-2e83-4034-a69b-1ae764a52107 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:37.995283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:37.996078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:37.996355 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:37.996661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.009655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.011150 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.023664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb35d5-7bb5-4bc9-9a10-ef0367f28cde None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:38.028894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.029946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.031232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:38.031232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:38.031232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.031232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:38.032836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 259/517] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.033695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.070899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.071756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.072160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.072569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.083139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.086093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.105081 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:38.110454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.111604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.112892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:38.112892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:38.112892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.112892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:38.113814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 259/518] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.115474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.129312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.162269 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f54713-1648-4b14-8d2a-f920080fd7c9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.203048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.203983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.204450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.204956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.205857 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:38.206162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.207567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.224697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 260/519] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:38.224697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.235038 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2339989-d8c9-4387-abb2-a1e1a49a49d4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.275721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.276358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.276676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.276954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.339287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f45bcc-5192-4048-8230-1619281d2bd2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.360731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.362602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.371671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.376220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.389440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.390103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.390392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.390667 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.391393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:38.391701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.393103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.412753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 261/520] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:38.413813 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.487874 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 260/521] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:38.488552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.533504 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0da5-d805-47ba-8568-56daa2be8c93 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.571945 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.572760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.573044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.573323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.574054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:38.574363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.575733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.592255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 262/522] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:38.592255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.623612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe50e9-cf79-407b-9858-8c450f2cb3b3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.638595 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.664927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.665878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.666162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.666437 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.674123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.675607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:38.690883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.692094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 261/523] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.712129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f5b0b-050a-4be3-aba2-fadfefad8558 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.752443 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.753251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.753534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.753811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.761890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.763605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:38.780281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.781260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:38.782212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 263/524] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.783379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.816072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c754ac-7ece-4993-a8e9-bf414896585a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.859672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.860836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.861399 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.861671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.874811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.876665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.894522 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:38.897667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.898528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.899374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:38.899374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:38.899374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:38.899374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:38.899687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14635e1-7ce7-4d01-a3fc-8507c703808f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:38.899756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 262/525] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.900370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:38.934460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:38.945805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:38.946609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:38.946896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:38.947885 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:38.954856 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:38.956323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:38.979028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:38.979949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 264/526] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:38.981601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.037382 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694601a-b53a-4d73-bc82-2c03a206a6ed None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.074116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.084487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.106873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 263/527] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.106873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.110102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d3963-77cf-4252-903d-216e0fd2e9f0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.148728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.149397 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.149677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.149947 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.150782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.151087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.152539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.170607 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 265/528] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:42:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.171423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.194092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.204789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.224318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024aa5cf-0283-4c9c-a973-d7390e590a2b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.263293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.263950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.264233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.264533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.265252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.265556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.266925 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.278776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 264/529] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.280005 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.289647 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403f4e-fabc-4e57-9b79-388b5e816037 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.335882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.336940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.337449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.337944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.338767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.339057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.340526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.360325 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 266/530] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.361478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.404070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370c5fa-ac6d-4ea0-a802-f349a000daa4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.441025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.441841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.442126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.442404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.443981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=2b920de9c0804a61832b483220501d04) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.444356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.446016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.460432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 265/531] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/role_assignments?user.id=2b920de9c0804a61832b483220501d04&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.460432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.462800 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.479306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.515821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8c25-abff-4d90-b8e9-4d2f2de1afe9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.569988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.570654 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.570937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.571212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.586735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c6cad-383f-491b-b795-5def19780d83 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.647021 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.648314 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.648779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.649184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.659463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.661785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.685136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:39.685922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:39.686899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:39.687921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:39.689084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.690008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 266/532] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:39.690970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.780701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.797833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:39.838549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd23d60-7107-46d5-a447-df1ad5954745 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:39.839268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 267/533] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:39.840540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:39.879883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:39.881313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:39.881975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:39.882496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:39.884133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=84e880ccf2cb4595abb3d43a7c026e76) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:39.884702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:39.887418 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:39.901044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 267/534] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/role_assignments?user.id=84e880ccf2cb4595abb3d43a7c026e76&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:39.901044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.034787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3b84b-be3b-42cc-b8e5-40eb2cc93861 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.086246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2c6b5-998d-45ee-a0c7-fa075fa8c1e6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.092162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.092817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.093099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.093373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.094171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:40.095203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.096732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23792-2107-48da-b1e3-30d3999ee369 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.115452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 268/535] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:40.117447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc1e5-a9f1-472c-9b93-eb98b1473d37 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.117734 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc1e5-a9f1-472c-9b93-eb98b1473d37 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.118008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffc1e5-a9f1-472c-9b93-eb98b1473d37 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.125534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 269/536] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:40.125751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc1e5-a9f1-472c-9b93-eb98b1473d37 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.129451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.137303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.138106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.138385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.138653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.151400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.152849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b67fb2e3-b7d1-4ed3-b822-e4093b91b1f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:40.187670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:40.189118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:40.190420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:40.190420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:40.190420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:40.190420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:40.191886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.198127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 268/537] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:40.198127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.198127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.198127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.227010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:40.229381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:40.292621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffc1e5-a9f1-472c-9b93-eb98b1473d37 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.316708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 269/538] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:40.318403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.339055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.340068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.340608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.342736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.343810 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:40.344331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.346560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.355243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 270/539] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:40.356499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.462350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b10ac7-0598-43a4-b2ed-c401393b0f9a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.473471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694190b-c5ad-4934-9df8-b99f30ce8c5f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.476082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.484842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.520888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.522407 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.522961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.523487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.524565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:40.525106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.526794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.531802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.532754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.533033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.533301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.548474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:40.548966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 270/540] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:40.550524 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:40.556983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:40.557924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:40.558699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 271/541] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:40.563905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.682179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f78bf9-6e23-42fb-bb70-0868e48c3959 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.686301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e735798-5651-4e32-a679-a33668b95c5c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:40.726066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.726769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.727066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.727347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.728268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:40.728602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.730226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.730493 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:40.731511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:40.732076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:40.732598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:40.754685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:40.757172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:40.766804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 272/542] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:40.773038 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:40.776380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:40.777246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:40.778088 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:40.784630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 271/543] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:40.790070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:40.844165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.872228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:40.970219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1be232-6e96-4b5b-8d33-b1895a326dcb None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.022675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.023346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.023626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.023899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.024712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:41.025024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.026500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.047887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42be99-1b73-4256-84a1-2ae68f84ac67 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.050389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 272/544] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:41.053821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:41.165012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:41.176623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c1c7b-a694-4f90-ac58-b1b1420d210f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.181127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.181940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.182226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.182507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.211066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.212562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.223979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.227169 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.227713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.228244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.229627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=e56d824c48c74091a66af4d24459c046) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:41.231735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.233424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.252795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 273/545] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:41 2022] GET /identity/v3/role_assignments?user.id=e56d824c48c74091a66af4d24459c046&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:41.257763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:41.258737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:41.259651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:41.259651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:41.259651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:41.259651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:41.262012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 273/546] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 489 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:41.269097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:41.440460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed86ad4-69eb-409a-8bb0-70caa213144f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.497727 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.498326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.498603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.498873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.499600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:41.499905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.501340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.515678 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 274/547] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:41.521829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:41.665242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:41.707400 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16767ab-786d-4073-ae68-5c3fdb145fe5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:41.762363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:41.763492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:41.763981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:41.764496 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:41.778946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:41.780759 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:41.800124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:41.801621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:41.803506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:41.805921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 274/548] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:41.811517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:41.856864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:41.961997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6448e33-2a7f-4238-bda7-c84b4a941d19 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:42.016387 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:42.017128 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.017462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.017740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.018659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:42.019024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:42.021135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b912f7b-14ed-4617-9114-9cc93f1139fe None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:42.050938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 275/549] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:42:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:42.056905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:42.201879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26279362-1234-4540-b3d7-9197f8085e20 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:42.226730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd670368-676d-44e3-a1f9-1dd1d12a80e6 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.227050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd670368-676d-44e3-a1f9-1dd1d12a80e6 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.227291 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd670368-676d-44e3-a1f9-1dd1d12a80e6 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.228682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 276/550] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:42.234187 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.234475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.234752 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.269856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:42.271533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:42.275894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:42.276579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.276866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.277144 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.285588 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:42.413952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 277/551] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:42.429742 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:42.431211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9517af8-b1fd-498e-8a25-d3e0e226c7a8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:42.449092 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:42.452193 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:42.595850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af093a0-5b84-43aa-8906-d54f74170ea6 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:42.621825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 275/552] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:42.624812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca08fb-7cf3-412d-bd0e-a84770a9e1dd None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.625455 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca08fb-7cf3-412d-bd0e-a84770a9e1dd None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.626272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca08fb-7cf3-412d-bd0e-a84770a9e1dd None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.632905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 276/553] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:42.634828 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca08fb-7cf3-412d-bd0e-a84770a9e1dd None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:42.644963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:42.645806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.646091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.646379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.658298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-70cbcb88-554a-491f-899d-5fada23cd65a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:42.667586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:42.668545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:42.669982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 278/554] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:42.670676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.670936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.671212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.687445 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:42.692455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:42.694740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:42.784823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca08fb-7cf3-412d-bd0e-a84770a9e1dd None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:42.786958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:42.804657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 279/555] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:42.805942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:42.832502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:42.833919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:42.834199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:42.834458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:42.848484 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:42.851256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:42.874513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:42.875446 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:42.876294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:42.876294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:42.876294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:42.876294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:42.878823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 277/556] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:42.880279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:42.949464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003cb301-55b1-4ccc-882a-03f01192615e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.002067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.003915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.004357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.004929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.006006 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:43.006487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.007401 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.009012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.022121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 280/557] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:43.023269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.027723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533a882-6def-4cd3-a3f3-20a9b468ae11 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.068527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.070035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.070457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.070871 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.082198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:43.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:43.092756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:43.093587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:43.095808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.097748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 278/558] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:43.100143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.146976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e5146-a379-4e80-a951-7ceb5e3aafc3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.184789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.185594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.185959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.186301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.187238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:43.187665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.189254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.207085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 281/559] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:43.208572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.225498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241a710-c8a0-4916-8bf0-967ee25decf4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.268338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.269164 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.269746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.269746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.281693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.283634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.294396 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-3cbbe9e6-ca8a-43e2-968b-e24592e6e861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:43.306322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:43.307377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:43.308292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:43.308292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:43.308292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:43.308292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:43.308567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 279/560] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:43.309763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fa61-ebb9-4fbe-8e98-91effb2eb6cf None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.310052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fa61-ebb9-4fbe-8e98-91effb2eb6cf None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.310327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fa61-ebb9-4fbe-8e98-91effb2eb6cf None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.311711 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 280/561] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:43.313147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5fa61-ebb9-4fbe-8e98-91effb2eb6cf None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.336274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3f480-a2b8-4fa3-ac10-cd30ea71a10d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.355149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.375573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.376245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.376564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.376840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.378027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:43.378352 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.380288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.393049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 282/562] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:43.393049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.436036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5fa61-ebb9-4fbe-8e98-91effb2eb6cf None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.476669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.477471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.477748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.478016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.485443 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f9167141-debe-4550-be1a-1781b904d624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:43.503551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:43.504624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:43.506601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 281/563] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:43.507595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.507595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.507595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.514829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a342d-16dc-485c-aaba-b336d6230086 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.546988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:43.548347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:43.551763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.552508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.552989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.553269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.562582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:43.562977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.564453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.578258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 283/564] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:43.579084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.624391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.637034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.637571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 282/565] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:43.638374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.727480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2cf4a8-72be-463d-94d9-24ac64a5d1ba None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.753649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d5895-a57e-45bd-ae4a-184681b81c69 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.765172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.766009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.766358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.766638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.780320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.781858 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.795688 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.796360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.796672 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.796948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.797752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:43.798062 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.799556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:43.802060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:43.803027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:43.803914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:43.803914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:43.803914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:43.803914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:43.803914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 284/566] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:43.804761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.823987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 283/567] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:43.823987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:43.878818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.916733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:43.930229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f1b2e-ab22-4b87-b05f-4c0eef84e890 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.937084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c746b05-22a5-4222-b18e-390a854781f4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:43.966941 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.967716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.968323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.968323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] PATH_INFO: `/v3/users/glance` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.977872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:43.979436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:43.981452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:43.982835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:43.983115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:43.983390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:43.994151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:43.995138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:43.996221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 285/568] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:43.997664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:44.004509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:44.006428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 284/569] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:44.007499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.102312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545aa241-1355-431c-8d15-93689ef4e1ae None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.110885 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659864fb-1ac1-4c72-90cd-1d5cc15988b2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.123451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.137746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.138332 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.138606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.138875 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.139596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:44.139901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.141361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a2cb9-293d-43b2-aa8f-2dabe7c0c0f3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.153304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.153730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 286/570] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:44.153964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.154250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.154528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.155002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52a10-d178-4dbb-b7c1-4caad7362ded None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.155289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52a10-d178-4dbb-b7c1-4caad7362ded None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.155385 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=433e683c78a242a695acebce7545f8cf) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:44.155564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca52a10-d178-4dbb-b7c1-4caad7362ded None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.155660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.156969 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 287/571] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:44.157160 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.157474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca52a10-d178-4dbb-b7c1-4caad7362ded None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.165255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 285/572] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/role_assignments?user.id=433e683c78a242a695acebce7545f8cf&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:44.166478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.173405 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.285473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca52a10-d178-4dbb-b7c1-4caad7362ded None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.310412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8df19-c0d1-4df5-a898-429bbd04fbe0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.333458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.334528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.334989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.335995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.337072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:44.337580 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.340592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c61f84b-9d41-48c4-b14e-76459b099065 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.352060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.352772 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.353054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.353946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.354065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:44.354368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.355736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.370869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 286/573] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:44.370869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.371285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 288/574] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:44.373001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445a00-a787-4094-b8d3-07b902594181 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.373298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445a00-a787-4094-b8d3-07b902594181 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.373577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79445a00-a787-4094-b8d3-07b902594181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.390915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-79445a00-a787-4094-b8d3-07b902594181 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:44.392198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-79445a00-a787-4094-b8d3-07b902594181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:44.408534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.462988 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.472457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 289/575] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:44.472457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79445a00-a787-4094-b8d3-07b902594181 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.489777 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9541120-507f-4f6f-9a68-295a776e9914 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.529937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.531478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.532419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.533067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.542143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.543579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:44.559495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:44.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:44.561783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:44.565389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 287/576] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:44.566153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.612666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79445a00-a787-4094-b8d3-07b902594181 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.656703 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.657939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.658359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.658764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.671805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.673627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:44.694148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:44.695244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:44.696295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:44.696295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:44.696295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:44.696295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:44.701428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 290/577] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:44.702460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.716282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.727611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96a1f-bf9b-47e8-aa13-2d4d5922edc1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.770312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.771556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:44.772471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.772754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.773025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.784784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.786551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:44.805881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:44.806875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:44.807800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:44.807800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:44.807800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:44.807800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:44.807800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 288/578] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:44.808660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.839029 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cad0a-bb3e-4520-a922-7961fa4c6266 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.881415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.882335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.882518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.882760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.894167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:44.902712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:44.903671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:44.904529 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 291/579] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:44.905617 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:44.939356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188e99fd-3c00-491b-a930-765a7b364127 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:44.991143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:44.992503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:44.993204 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:44.994250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:44.995651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:44.996411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:44.998708 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.031241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 289/580] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.031241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.038951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.048975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7c6bd6-352a-4c3e-83c2-e6a023400a68 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.064010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.100637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.101294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.101725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.101873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.102718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:45.103533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.104260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.125523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 292/581] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.125523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.161571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c19c1-ef3d-4d5d-a467-f1978f81e84f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.201063 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.201625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.201914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.202191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.202927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:45.203261 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.204643 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.221723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 290/582] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.221723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.268526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020880f2-8cc8-439b-a61d-f609ae2d86c8 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.298573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.313135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.313811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.314259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.314642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.315591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:45.315965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.317662 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.333490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 293/583] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.333490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.349002 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47969819-9ada-41c1-a700-d8408a2d1b17 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.376612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.393818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.394615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.394892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.395187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.408147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.409631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:45.433876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:45.434849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:45.435737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:45.435737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:45.435737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:45.435737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:45.435737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 291/584] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:45.436019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.477905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f93deb-431f-412d-9b9e-e14347568e88 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.520465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.521245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.521530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.521804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.530700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.532205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:45.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:45.549706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:45.550707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 294/585] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:45.551303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.576153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743cbb7-a2cf-494a-9ae1-199b7700cd16 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.606040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.625192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.626119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.626527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.626805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.634363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.635836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:45.651373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:45.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:45.653285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:45.657494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 292/586] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:45.658390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.669639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.681428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb3fb-7d8d-4fc3-9a2d-33eeeca65b77 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.736741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.737463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.737745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.738019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.750839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.752482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:45.771072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:45.772167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 295/587] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:45.773067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.779295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670accb3-5ff6-4878-856b-8e856254fe60 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.830802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.831759 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.832042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.832315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.833625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:45.833912 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.835729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.856012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 293/588] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.856012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.887911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aa53-c464-45b4-950b-f9a5b04204df None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:45.910376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.933427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:45.934099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:45.934379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:45.934659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:45.935483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:45.935798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:45.937262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:45.957564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 296/589] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:45.959976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:45.964565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:45.989895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb361b68-a05f-4bbd-92af-2345ea5763ca None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.027630 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.028286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.028599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.028879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.029655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:46.029964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.031383 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.044145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 294/590] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:46.045092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.085295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc405-dbc1-4416-811a-7f33856573c9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.127769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.128455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.128721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.128988 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.129761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:46.130061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.131446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.144298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 297/591] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:46.145669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.167938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.172280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461893e7-533c-4b69-afd9-69617b9fef86 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.214887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.215704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.215988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.216269 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.225232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.226703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:46.243414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:46.244786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:46.245673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 295/592] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:46.246149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.253115 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.297518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08a1bf-7aef-4194-bac0-c9cc9d47c894 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.338606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.339433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.339718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.339996 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.353069 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.354605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.366726 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d094d7-9bad-418e-82c2-d0b7578735e7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:46.375892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:46.376935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:46.377801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:46.377801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:46.377801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:46.377801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:46.383196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 298/593] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:46.383918 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.411381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.412608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.412894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.413169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/2b920de9c0804a61832b483220501d04/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.431540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=2b920de9c0804a61832b483220501d04, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.433193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.460503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 296/594] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:46 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/2b920de9c0804a61832b483220501d04/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:46.461779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.481441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.511340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b4e67-f78f-479f-bd8e-754eb405c3a4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.540562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.567866 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.569005 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.569281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.569554 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.584016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.585557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:46.606028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:46.607281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 299/595] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:46.608243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.624227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee992f-861f-43c1-949a-d07dba2ee870 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.678889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.679659 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.679946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.680225 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.681054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:46.681361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.682880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcf3b2-2350-42bc-9849-b2b6b0974771 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.701714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 297/596] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:46.701714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905de6-e87e-4c5a-b88b-08744bc84df1 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.701714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905de6-e87e-4c5a-b88b-08744bc84df1 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.701714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75905de6-e87e-4c5a-b88b-08744bc84df1 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.701714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 298/597] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:46.702265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905de6-e87e-4c5a-b88b-08744bc84df1 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.732248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3405ca1-d334-44bb-819e-e050c319160d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.774858 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.775582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.775712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.775991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.776742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:46.777049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.778137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.778429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.803538 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 300/598] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:46.803538 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:46.825291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75905de6-e87e-4c5a-b88b-08744bc84df1 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.837678 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:46.876958 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.877742 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.878096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.878379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.879232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:46.879604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.881680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0be1e-07ed-4082-9edf-259d110e29b3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:46.910754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 299/599] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:46.917528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.917804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.918085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.923207 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa238f-bafe-4ccd-9169-9c008055f879 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:46.940451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:46.941825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:46.961869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:46.962660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:46.962942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:46.963350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/84e880ccf2cb4595abb3d43a7c026e76/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:46.981266 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=84e880ccf2cb4595abb3d43a7c026e76, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:46.982746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.008430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 301/600] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:46 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/84e880ccf2cb4595abb3d43a7c026e76/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:47.008430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.045716 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 300/601] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:47.046515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.129944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.147352 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.156761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1ec9d-2865-4a25-840f-1719228a435c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.207486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.207486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.207486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.207486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.215950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.217485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.230541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54f51-4399-497d-bfb4-a83ec8666af9 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:47.234310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:47.235208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 302/602] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:47.236077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.280093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.281312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.281802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.282365 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.295811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.297739 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:47.318922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:47.319873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:47.320959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:47.320959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:47.320959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:47.320959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:47.323544 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 301/603] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:47.324675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.381549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24bb8f-6634-4679-9a1d-cafa4ec6fd46 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.419045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.420262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.420867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.421372 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.434166 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:47.443033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:47.443994 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:47.444846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:47.447780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 303/604] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:47.448826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.450508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57274d-9caf-4752-9c3d-c358d8e37a9e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.453959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.462832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.497126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.497792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.498074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.498344 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.499126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:47.499432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.501286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.520913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 302/605] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:47.520913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.580091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538bb55-de05-4d5d-940a-96c3167ee745 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.622002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.622730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.623067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.623439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.624259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:47.624776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.627401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.648483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 304/606] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:47.649719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.663099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8b93d-3b63-4643-9102-20fcc046f6b1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.710505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.711173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.711451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.711725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.712837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:47.713158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.714539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.725017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 303/607] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:47.726041 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.727300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.763555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:47.771256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb3527-e5a9-4aba-b6d5-02668dac6849 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.811677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.812651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.812991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.813273 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.829498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=6196d18cd1b44984a2b1c0602745314f, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.831289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.849336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639425b1-3aa7-45b4-a791-057273c83c76 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:47.850350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 305/608] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:47 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:47.851655 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:47.897067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:47.898335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:47.898885 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:47.899490 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:47.909015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:47.911929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:47.930017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:47.931078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:47.931990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:47.935031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 304/609] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:47.943951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:48.040556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05227922-b992-44c7-a620-f0bbb771b825 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.061589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:48.106183 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.106913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.107195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.107468 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.129017 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.131192 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.142650 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:48.154335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e2a6b-478b-410c-9324-9e69a59bd5dd None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:48.160308 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:48.161277 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:48.162361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:48.162361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:48.162361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:48.162361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:48.162361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 306/610] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:48.169911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:48.199678 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.201033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.201526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.201952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.203290 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:48.203758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.205898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.219982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 305/611] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:48.225511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:48.350615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c709f7-95cd-4886-8689-bd681dac1f7c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.390481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.391776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.392287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.392952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/e56d824c48c74091a66af4d24459c046/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.414244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=e56d824c48c74091a66af4d24459c046, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.417238 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.446113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 306/612] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:48 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/e56d824c48c74091a66af4d24459c046/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:48.447301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:48.480481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b364b3-abba-4551-b58f-fd082cc89be7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.524754 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.525834 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.526397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.526965 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.527979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:48.528536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.530321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.548874 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 307/613] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:48.550782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:48.674106 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863d49-3842-40dd-a336-c495c7a55af9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.711804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.713026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.713519 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.714019 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.722779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.725351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:48.742343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:48.743596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:48.744582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:48.747168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:48.753011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:48.785171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:48.865562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d593cc6-8697-4664-878f-816460cf6e00 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:48.927437 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:48.928060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:48.928345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:48.928654 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:48.929523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:48.929836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:48.931400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6740ab-36a1-4e7a-91ad-0b1817a328b9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:48.958978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 308/615] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:42:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:48.964259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.066760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.111474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539d4bf-2ab3-4d94-8b21-86e36bc8d3a5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.126821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14157-50d6-46d2-acc3-4d58fdd15067 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.127491 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14157-50d6-46d2-acc3-4d58fdd15067 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.127984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b14157-50d6-46d2-acc3-4d58fdd15067 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.129939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 309/616] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:49.135184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.135796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.135796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.151588 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.152393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.152702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.152985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.165442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:49.166905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:49.172785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:49.173976 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:49.187700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:49.188638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:49.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:49.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:49.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:49.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:49.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 308/617] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:49.195990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.264891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 310/618] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:49.297042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.319416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.376635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4829347-1801-4753-94ef-78d6ca632c46 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.392824 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.419710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.420393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.423349 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.423644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.425449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:49.425769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:49.427472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:49.457528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 309/619] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:49.461681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.518180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf45e9f-5ada-47e7-95b5-c3dadaead00b None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.593093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.593906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.594190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.594465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.625900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:49.638402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:49.639391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:49.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 311/620] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:49.647777 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.699585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dde3c-6d12-4ead-b634-ad9be6e2884a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.737079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.740868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.741867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.742161 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.742441 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.750283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:49.751741 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:49.765142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad0fc7-ba9e-4545-89e5-210f07b90d23 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-303699d5-3e7d-4c5a-8f46-39d41c580e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:49.767391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:49.768533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:49.769620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:49.772863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 310/621] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:49.774329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288c48-fb01-4f04-b054-4b7a14da92e1 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.774620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288c48-fb01-4f04-b054-4b7a14da92e1 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.774901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288c48-fb01-4f04-b054-4b7a14da92e1 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.781075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 311/622] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:49.781886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288c48-fb01-4f04-b054-4b7a14da92e1 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.807007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.807834 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.808182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.808607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.809463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:49.809841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:49.811321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b820d3-8b2d-4568-ba11-d72fafa05b76 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:49.822253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 312/623] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:49.822936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.824043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.824438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.824870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.842559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:49.843861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:49.903077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288c48-fb01-4f04-b054-4b7a14da92e1 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:49.918358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 313/624] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:49.919507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:49.971229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:49.972022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:49.972307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:49.972706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:49.974134 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:49.974451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:49.976008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:49.996535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 312/625] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:49.996926 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:49.998591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.035182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c041-29fd-4fb4-81f9-b4db327120c9 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.073672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.074615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.074902 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.075178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.087372 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.089042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.107002 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:50.108109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:50.108997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:50.109856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:50.109856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:50.109856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:50.109856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:50.109856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 314/626] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:50.110544 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.124071 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d200e-c0a6-4a8b-8b92-5021b3b8c60d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.164878 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.165683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.165964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.166240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.178841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:50.187423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:50.189615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:50.191564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:50.200425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 313/627] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:50.200425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.223844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efee70d-487b-4e36-a9a2-179859a891b7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.244538 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:50.263426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.264522 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.264870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.265212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/433e683c78a242a695acebce7545f8cf/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.283204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=433e683c78a242a695acebce7545f8cf, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.284876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.300922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 315/628] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:42:50 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/433e683c78a242a695acebce7545f8cf/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:42:50.309838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.312009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d7dc-9f13-4d43-87e5-151c98252055 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.363221 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.365120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.365406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.365685 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.366448 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:50.366758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.368198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 314/629] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:50.405058 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.437205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:50.471848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea1378-eba8-4739-ab55-452d27fe3ef6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.538675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.540561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.541505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.542012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.543869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:50.544648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.549028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.563020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 316/630] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:50.569118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.600286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d996-4cb2-43d7-b3cf-0f6afd5a02c9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.620034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:50.663239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.664664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.665147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.665649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] PATH_INFO: `/v3/users/glance` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.678948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-24a6995e-a753-4571-b527-e13a312c037c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:50.697937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:50.698955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:50.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 315/631] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:50.701506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37698b-bbe6-487e-b0e4-a6e44eae2bf6 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.701897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37698b-bbe6-487e-b0e4-a6e44eae2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.702276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37698b-bbe6-487e-b0e4-a6e44eae2bf6 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.704488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 316/632] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:50.705524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37698b-bbe6-487e-b0e4-a6e44eae2bf6 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.714494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989880a5-d6d5-47db-85b0-bce1d89bc3d3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.754736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.755936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.756379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.756851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.769393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.771247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-4128326a-42ae-4377-9af8-b0c60871fd82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:50.791432 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:50.792328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:50.793220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:50.793220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:50.793220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:50.793220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:50.795229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 317/633] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:50.796621 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.797164 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.797866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:50.798729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.819301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:50.820558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:50.847051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37698b-bbe6-487e-b0e4-a6e44eae2bf6 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:50.893049 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:50.894093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:50.894582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:50.895666 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:50.896808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:50.897397 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:50.899160 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:50.912903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 318/634] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:50.913954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.922422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 317/635] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:50.933471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:50.934016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.042319 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f985-f4ca-4730-ab5c-1b4067fe6ecd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.069356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fffd3-1748-4442-8cef-2ff76a886117 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.083442 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.084380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.084833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.085249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.086117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:51.086589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.088131 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.100959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 319/636] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:51.102121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.116503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.128796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.130314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.130597 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.130867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.190080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.210706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.212152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880562b-7ff5-4539-ad3e-6371c1b2e9c7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.226768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bcd71d-f772-4793-bbb1-55067409f104 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.269653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.271479 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.271758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.272297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.286946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:51.294827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:51.295745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 320/637] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:51.302279 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.328591 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 318/638] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:51.330336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dd1-e21e-443b-82a2-6e2a2c650275 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.330890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dd1-e21e-443b-82a2-6e2a2c650275 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.331173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c97dd1-e21e-443b-82a2-6e2a2c650275 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.333676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 319/639] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:51.334272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c97dd1-e21e-443b-82a2-6e2a2c650275 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.367483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.424289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0df84-43a1-4388-8410-ae87fc9d204a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.450198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c97dd1-e21e-443b-82a2-6e2a2c650275 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.454870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.461105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.461909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.462198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.462473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.470388 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.471838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:51.487095 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:51.488132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:51.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 321/640] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:51.494043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.494197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.495020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.495469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.495902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.497142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:51.498508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.500036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281343f5-3855-423e-99a5-8a6a5abd4d14 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.514931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 320/641] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:51.514931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.514931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.514931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.529178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:51.530388 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:51.616512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 321/642] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:51.616512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.625694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f025140-5d88-4473-a835-9a1c52887762 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.641858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.666484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.667560 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.668061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.668515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.681833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.683718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:51.702859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:51.703883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 322/643] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:51.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.735176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e77f71-0615-40f4-a00b-431681797055 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.758257 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.777813 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.778479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.778758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.779029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.779865 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:51.780171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.781729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.802313 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 322/644] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:51.803302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.832830 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae873580-8c6b-48c8-881a-90bc691228e7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:51.877741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:51.878477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:51.878730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:51.879019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:51.890370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:51.891828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:51.905562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:51.912456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:51.913329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:51.914374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:51.914374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:51.914374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:51.914374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:51.914374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 323/645] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:51.914651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:51.934197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66ec6a-de6d-48b4-b6b3-3b9b1178f4e0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.006665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.007608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.007818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.008050 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.020379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:52.028861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:52.030013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 323/646] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:52.036505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.039524 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.046196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c6010-3e19-44cd-a2f2-af65348655ee None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.095719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.096609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.096956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.097373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.098638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:52.098942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.100325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.119321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 324/647] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:52.121101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.173149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f7bdd-78f8-4f50-848b-aeec5e0d6cfb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.211790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.212582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.212929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.213510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.226375 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.234548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10952c0c-5c3b-42e3-83f1-f67abdd653b0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.273557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.274621 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.274900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.275167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.275888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:52.276190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.277614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.295815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 325/648] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:52.295815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.308485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.310898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.315237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.417332 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd345e-04d8-40fa-95a1-6e181f1f6b46 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.422684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 324/649] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:52.422684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.461386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.461985 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.462269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.462546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.463299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:52.463741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.465280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.466422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.482828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 326/650] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:52.482828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.536734 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c9752-c2d1-46b5-bdd8-71fa8b926e00 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.569665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.573724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.574563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.574839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.575108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.583299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.584804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.598739 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ade653-65d5-47de-9ce1-8371362809a1 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:52.599293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:52.600148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 325/651] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:52.606016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.649606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.650429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.650712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.651007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.660446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.662080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:52.678406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:52.679426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:52.680378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 327/652] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:52.681466 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.733541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.737844 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298b053-03f6-494e-b370-d29706b0a525 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.777276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.778068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.778354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.778631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.792927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c341e02-386c-4e5b-a836-dfcf83ccd77d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.795872 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.797461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:52.816166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:52.817906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 326/653] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:52.821036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.837652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.838477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.838759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.839027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.846562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:52.852700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.854191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:52.872313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:52.873265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:52.874134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:52.874134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:52.874134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:52.874134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:52.874134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 328/654] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:52.874769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:52.937252 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f2-9610-4161-8f12-3a64f7f9ac6f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:52.975007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:52.975671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:52.975954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:52.976230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:52.977075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:52.977387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:52.978818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:52.992816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 327/655] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:52.994173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.001159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35076b39-07df-4013-84a4-9ebd9df94ba0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.011475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.039899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.040598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.040879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.041154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.048230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:53.048577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.050090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8b521-1dea-44ce-8cc3-1c22dd3514c1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.069346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 329/656] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:53.070862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a052b-ef91-4632-8484-44ca771ce674 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.071158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a052b-ef91-4632-8484-44ca771ce674 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.071435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a052b-ef91-4632-8484-44ca771ce674 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.077163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 330/657] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:53.077163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a052b-ef91-4632-8484-44ca771ce674 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.105172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeeef35-a912-4463-9c21-e801f07bbc73 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.125722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.141727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.142380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.142659 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.142932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.143638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:53.143946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.146137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.158406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 328/658] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:53.159251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.186052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a052b-ef91-4632-8484-44ca771ce674 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.225705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.226365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.226649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.226919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.227629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:53.227934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.229336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.235158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.241397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 331/659] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:53.242243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.270202 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cdc5e-f503-4ec6-afe6-52b36c8a39d3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.307214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.308353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.308926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.309346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.310709 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=2b920de9c0804a61832b483220501d04) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:53.311183 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a03057-7e41-4887-b188-bc62e2d1ea64 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.319597 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 329/660] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/role_assignments?user.id=2b920de9c0804a61832b483220501d04&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:53.320852 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.321141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.321415 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.340072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:53.341467 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:53.361126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab8c03-e36d-475a-9f24-332c1adde15d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.372520 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.406149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.406846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.407124 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.407393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.428338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 330/661] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:53.429270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.496729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.498208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.530419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.571971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b750cdf-30a8-493f-8873-2be27725ce75 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.615314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.616155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.616472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.616811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.625304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.626817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.641528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 332/662] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:53.641528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:53.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:53.643617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:53.645073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 331/663] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:53.645569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.655603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.761217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffcb92-f7e4-4d44-aa05-ae770b515bdc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.770298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802d3c-f21a-41e0-82ec-336c0635aca3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.798158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.798871 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.799147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.799416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.799635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.808265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:53.809211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.809852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:53.810051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.810370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.810738 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:53.822839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:53.825077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:53.826185 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:53.827154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 333/664] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:53.827808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:53.836085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:53.837428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 332/665] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:53.838436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:53.913744 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:53.954078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018ee9-1a16-46be-b21e-e5683838b3e4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.961382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f906b6-47cf-4317-9ab8-c74e51ad048d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:53.996779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:53.997559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:53.997836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:53.998102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.007773 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.008551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.008751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.009028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.009815 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.010000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.010158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.011433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.011541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.024958 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 333/666] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.029316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:54.030760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:54.031747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:54.032652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:54.032652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:54.032652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:54.032652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:54.038051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 334/667] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:54.038246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.092832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.150157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1d19-f97a-4532-8101-5eebce845083 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.160927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65743e-37e6-4942-a43d-b3abf44a5e09 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.182718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.192074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.193058 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.193491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.193761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.194614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.194922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.196817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.202590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.203258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.203536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.203892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.204572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.204878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.206250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.219586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 335/668] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.219586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.222496 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 334/669] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.226722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.340711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c3477-7f0b-4a73-a8ea-87a6a076c670 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.358436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb3c16-0b39-43ff-8066-c09a9aa1259b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.361457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.378406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.379198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.379484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.379757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.380611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=84e880ccf2cb4595abb3d43a7c026e76) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.380929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.382373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.388486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 336/670] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/role_assignments?user.id=84e880ccf2cb4595abb3d43a7c026e76&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.389263 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.417735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.419087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.419608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.420672 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.422004 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.422532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.425299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.443389 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 335/671] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.445076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.448899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.521665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9049da-43c0-4cc9-b709-d30876bb2c09 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.564056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.565198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.565658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.566137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.574153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c2d3-d41f-4743-91c8-9f9329a33bb9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.579168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.580993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:54.600664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:54.601609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:54.602577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:54.602577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:54.602577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:54.602577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:54.604586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 337/672] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:54.606485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.615649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.616347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.616552 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.616866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.656200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.712062 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.713575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.730086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.735025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a912287-dd75-4293-971a-6ad89d5ab015 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.776690 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.777629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.777916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.778191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.786011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.787545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:54.803353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:54.804268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:54.805171 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 338/673] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:54.806102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.833477 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 336/674] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:42:54.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:54.902717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:54.924002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f4c1-cd0e-45fa-b519-e10e33a8621c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.945641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e835c0-5961-4eb5-a49f-1252da940915 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:54.963137 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.969822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.970340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.970770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.971814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.973618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.975487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.988137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:54.988870 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:54.989148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:54.989416 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:54.990205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:54.990571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:54.995105 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:54.995695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 339/675] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:54.996872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.005270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 337/676] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:55.006204 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.019260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.116531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3f24-15ac-47b6-a9b2-65ad15611855 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.121262 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8944c8-e82a-41d2-821e-fb85ecc3e7e2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.148564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.160653 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.160824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.161380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.161656 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.161930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.162574 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.163110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.163581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.170578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.172128 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.178416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.180272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:42:55.188437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:55.189451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:42:55.190377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:55.194775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 338/677] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:55.195740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:55.204209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:55.205132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:55.206022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:55.206022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:55.206022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:55.206022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:55.208726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 340/678] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:55.210395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.294890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.314005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9243d-6484-48ce-8faa-e2b113ee796c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.333718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442a6b-1001-4664-a2f4-24347167a882 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.354173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.355066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.355180 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.355459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.356296 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:55.356634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.358143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.365573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 339/679] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:55.366560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.375870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.377294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.377795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.378269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.379836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:55.380518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.382344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.413219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 341/680] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:55.423618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.433061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.513906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ee89a-6cee-43a8-b90e-f1c96e1bb8b3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.580386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b748fc-4191-4cd4-a88b-f61b0f933a57 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.590892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.591560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.591840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.592112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.592874 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:55.593282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.594706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.611336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 340/681] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:55.616992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.624698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.637574 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.638233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.638621 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.638897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.731959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.733824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c458aa-8809-4e7f-a67c-1f5deba86968 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.763230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09084c-6c5a-4c6b-a88b-ed1eaef9da38 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:55.782868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.828632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:55.829454 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.829886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.830182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.842475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:55.844311 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-da87b37b-356d-4630-9280-620532c1f7ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:55.865284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:55.866239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:55.867785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 341/682] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:55.868788 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f8c7-3c6d-404a-acd9-630ff67b3e6c None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.869070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f8c7-3c6d-404a-acd9-630ff67b3e6c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.869341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735f8c7-3c6d-404a-acd9-630ff67b3e6c None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.875391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 342/683] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:55.875391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f8c7-3c6d-404a-acd9-630ff67b3e6c None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:55.879743 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 342/684] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:42:55.881653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:55.882230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:55.882704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:55.901593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:55.903280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:55.930931 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:55.995244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 343/685] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:55.995548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.020942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735f8c7-3c6d-404a-acd9-630ff67b3e6c None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.060874 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.070774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.071056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.071330 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.072162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:56.072513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.074043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.094403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 343/686] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:56.094403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.136668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabb80b-aa91-44a9-84ca-89953b5b73bd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.156735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:56.180542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.181361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.181647 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.181922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.195023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.196570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.231205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a729c9a-f7ff-4c08-9766-bd295d34bd55 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.273023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.274103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.274548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.275006 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.288242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.290024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:56.311140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:56.312273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:56.313322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:56.313322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:56.313322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:56.313322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:56.319903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 344/687] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:56.321117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:56.346612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:56.348481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:56.349510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:56.349510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:56.349510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:56.349510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:56.349510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 344/688] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:56.349843 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.366923 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:56.419046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:56.443192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56070e1-5048-4bcb-b2e5-978b0e108401 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.464586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a2183-50dd-4c1e-afe2-3bba94a6bca3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.484193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.485152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.485578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.485992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.487105 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=e56d824c48c74091a66af4d24459c046) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:56.487560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.489986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.504817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 345/689] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:56 2022] GET /identity/v3/role_assignments?user.id=e56d824c48c74091a66af4d24459c046&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:56.505607 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.511480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.512167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.512464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.512738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.513483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:56.513793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.516299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.564905 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 345/690] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:56.573114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.674150 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:56.713996 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a6d2a-e1c7-498d-81f1-dda08c777604 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.766771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.767786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.768262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.768784 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.770318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.771967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.777593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3bb92f-dc9c-4265-b1bb-01fd49810a20 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:56.797281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 346/691] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:42:56 2022] POST /identity/v3/users => generated 254 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:42:56.823102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:56.824270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:56.824863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:56.825342 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:56.826454 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:56.826996 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:56.828816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:56.846101 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 346/692] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:56.851592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:56.981045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e202d20-7aa9-4fc0-b018-855cf60590f7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.012888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.023912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.024806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.025372 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.025694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.036249 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.037810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-40b93f16-bc4e-4bfa-b84c-82e885e4f6df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:42:57.054543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:57.055505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:57.056433 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 347/693] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:57.063409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:57.068341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.103781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8924-9349-4e6d-8ccc-ebd59a6280e3 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.104076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8924-9349-4e6d-8ccc-ebd59a6280e3 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.104248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b8924-9349-4e6d-8ccc-ebd59a6280e3 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.107268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 348/694] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:57.112111 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.112635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.112918 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.136988 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:57.138721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:57.255830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189fa-e646-4fce-8a8d-c0b73a7ef369 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.295812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.296507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.296791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.297066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.297858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:57.298078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 349/695] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:42:57.304726 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.306295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.325393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 347/696] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:57.331205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:57.339463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:57.417681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.465407 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb6259-3908-4db5-b854-5a613972df33 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.506704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.507532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.507809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.508105 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.510488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.510488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:57.526691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:57.527849 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:42:57.528724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:42:57.533898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 348/697] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:42:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:42:57.536021 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:57.615111 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a0099-73d3-46db-9738-2d4fc1f74440 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.654266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc8491-df2b-44e3-aef6-787c1709b09e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.663470 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.687358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:57.698208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.698971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.699255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.699532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.700245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:57.700580 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.701964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.706836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.707930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.708394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.708876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.710383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=433e683c78a242a695acebce7545f8cf) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:57.710916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.711764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 349/698] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:42:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:57.713177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:57.713489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3bd89-1cde-48a4-a4c1-dc73fb195ec6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.729731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 350/699] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:42:57 2022] GET /identity/v3/role_assignments?user.id=433e683c78a242a695acebce7545f8cf&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:57.839382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f43fd-bce8-4938-a146-b0a68bf2a79b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:57.883263 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:57.884012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:57.884295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:57.884602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:57.885379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:57.885692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:57.887094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e43c22-df70-4154-aa72-684abc7794e3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:57.900608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 350/700] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:42:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:42:58.337222 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd419cd-9927-4fa0-8c75-a893290c7a8e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:58.337222 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd419cd-9927-4fa0-8c75-a893290c7a8e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:58.337477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd419cd-9927-4fa0-8c75-a893290c7a8e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:58.339603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 351/701] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:58.345179 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:58.345472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:58.345750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:58.366794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:58.368268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:58.443127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:58.478250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 351/702] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:58.530322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd419cd-9927-4fa0-8c75-a893290c7a8e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:58.674027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd419cd-9927-4fa0-8c75-a893290c7a8e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:58.724289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:58.725120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:58.725395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:58.725664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:58.745492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:42:58.753233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:58.754245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:58.755061 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 352/703] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:42:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:58.765513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:58.937277 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dd0e3-d2ad-44ea-8c87-fa06fe8c283b None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:58.998717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:58.999409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:58.999687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:59.000244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:59.000743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:59.001047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:59.002495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:59.032447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 352/704] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:42:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:59.038693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:59.130969 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:59.268710 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0632b07-5df7-4d53-adc9-5eff5aaf7657 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:59.350337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:59.351168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:59.351446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:59.351715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:59.373753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:59.375245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ea612edb-9445-4d1d-971f-7b49da50975d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:42:59.414809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:42:59.415812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:42:59.416768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:42:59.416768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:42:59.416768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:42:59.416768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:42:59.416768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 353/705] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:42:59.422541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:59.558314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:59.574526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384963-710c-4757-a56c-d5a6adb2b18c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:59.574912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384963-710c-4757-a56c-d5a6adb2b18c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:59.575104 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1384963-710c-4757-a56c-d5a6adb2b18c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:59.578413 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 354/706] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:42:59.588562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:59.588887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:59.589115 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:59.628195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:42:59.632858 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:42:59.738378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478e598-4f38-4a6b-84b6-4e3aed987edf None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:59.809707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 355/707] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:42:59.819976 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:42:59.820680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:42:59.820959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:42:59.821233 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:42:59.821963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:42:59.822266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:42:59.823629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:42:59.837750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:59.842091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 353/708] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:42:59.847773 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:42:59.903540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:42:59.980543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55b065-8658-4358-9484-b3f173c4921f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:42:59.996994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.028515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.028515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.028816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.029064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] PATH_INFO: `/v3/users/nova` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.043196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.044716 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:00.072373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:00.073374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:00.074298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:00.079931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 354/709] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:42:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:00.084817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.104945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eea63-c713-4d35-afdf-c806025f2faa None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.145451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.146764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.147288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.147844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] PATH_INFO: `/v3/services/network` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.148864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.150383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-c00582c5-3b8f-46d2-aa2e-a7fd26ba9ba9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:00.165090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:00.166122 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:00.166978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 356/710] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:42:59 2022] GET /identity/v3/services/network => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:00.168632 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11d66-19a7-49b0-a5de-94c1a3e3d214 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.168986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11d66-19a7-49b0-a5de-94c1a3e3d214 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.169329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac11d66-19a7-49b0-a5de-94c1a3e3d214 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.174915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 357/711] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:00.174915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac11d66-19a7-49b0-a5de-94c1a3e3d214 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.237399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d01541-4b28-4712-a8dd-98a7d717329c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.278619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.279405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.279820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.280133 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.281072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:00.281491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.283503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd126e1-d92b-4509-a2d3-ad2adabd4e0f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.309250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.320133 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 355/712] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:00.321753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.322208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.322728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.335264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac11d66-19a7-49b0-a5de-94c1a3e3d214 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.345259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:00.346943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:00.384964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.388749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.389695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.390364 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.391182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.392388 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] RBAC: Adding query filter params (name=network) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:00.392940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.395918 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.402456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 358/713] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:00.403698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.456428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 356/714] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:43:00.456428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.551398 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c8958-b02c-46f3-9f5f-36041c5f384c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.593180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.594197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.594699 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.595175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.595590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2ca89-eac5-4f8f-87fb-c2e65fa91a53 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.618074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.643065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.645663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.646210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.646739 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.647812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] RBAC: Adding query filter params (type=network) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:00.648361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.650069 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.656332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 357/715] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:00.657748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.682517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:00.684064 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0332c-7247-4111-8dea-fc611ceb5484 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:00.692866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.803046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7bb99-b530-4118-bed3-15e7c96a5599 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:00.807465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 359/716] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:43:00.809236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097a57-f171-4f06-a0a2-4799cdde784a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.809795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097a57-f171-4f06-a0a2-4799cdde784a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.810750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097a57-f171-4f06-a0a2-4799cdde784a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.812700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 360/717] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:00.817939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097a57-f171-4f06-a0a2-4799cdde784a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:00.863771 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:00.864805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.865087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.865359 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.878702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-d40dcf8e-19b3-4d00-bd1c-7fb3cfbda567 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:00.886752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:00.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:00.888628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 358/718] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:00.889335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4e94-ded4-4a92-b273-a60fe078e7bc None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:00.890439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4e94-ded4-4a92-b273-a60fe078e7bc None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:00.890439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea4e94-ded4-4a92-b273-a60fe078e7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:00.904124 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:00.908665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-94ea4e94-ded4-4a92-b273-a60fe078e7bc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:00.910452 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-94ea4e94-ded4-4a92-b273-a60fe078e7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:00.975956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097a57-f171-4f06-a0a2-4799cdde784a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.003328 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 359/719] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:43:01.008664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b02cb-dcee-421e-bf36-161719d900d7 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.008664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b02cb-dcee-421e-bf36-161719d900d7 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.008664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b02cb-dcee-421e-bf36-161719d900d7 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.008664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 360/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:01.008664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b02cb-dcee-421e-bf36-161719d900d7 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.021421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.021952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.022310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.022656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.034170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.044360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.045865 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-8a4a39d3-bda3-4f87-a97e-6b92e4c538fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:01.065044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:01.066594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:01.067511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:01.067511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:01.067511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:01.067511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:01.067511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 361/721] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:01.068597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceed07-0718-40df-9991-d242d5250336 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.069066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceed07-0718-40df-9991-d242d5250336 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.069338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceed07-0718-40df-9991-d242d5250336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.087765 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-48ceed07-0718-40df-9991-d242d5250336 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:01.089145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-48ceed07-0718-40df-9991-d242d5250336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:01.143648 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b02cb-dcee-421e-bf36-161719d900d7 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.169123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 362/722] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:43:01.170242 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceed07-0718-40df-9991-d242d5250336 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.186886 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.187569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.187850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.188124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.188934 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:01.189242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.190632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.200711 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 361/723] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:01.201615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.217894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.287275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.308430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceed07-0718-40df-9991-d242d5250336 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.325014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46994b63-ffcc-4951-ac21-f62004cc0fa6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.354891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.355552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.355826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.356093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.356839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.358210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.373635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.374303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.374602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.374864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.375589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:01.375900 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.384797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 363/724] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:01 2022] POST /identity/v3/services => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:01.385001 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.385118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.400022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 362/725] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:01.400850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.520003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.541085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6d0ec-2641-40c6-949e-0920f45b021c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.578901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8557dbba-711f-4b7a-aedb-35d3fcd912f2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.587018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.588004 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.588381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.589434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.605782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.607631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:01.630454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:01.631389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:01.632270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:01.632270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:01.632270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:01.632270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:01.633297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 364/726] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:01.633537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.634121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.638380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.638380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.638613 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.638893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.664818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:01.672620 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:01.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:01.674499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:01.676677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 363/727] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:01.677859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.780689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cc08b-85fc-4f35-8523-c1f163944f1f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.804689 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa7fd-d12f-4e56-a444-a35f31563b48 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:01.823598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.825290 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.825544 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.825782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.834235 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.835717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.838440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.852341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:01.852596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:01.853409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:01.854299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 365/728] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:01.855302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.856199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:01.856742 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:01.857431 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:01.858629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:01.859167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:01.860855 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:01.878206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 364/729] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:01.879999 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:01.942977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:01.991022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac11f-f22b-4ee3-89e6-9b725890ac27 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.005968 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f2159-d9a1-457e-8754-32a343981643 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.031365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.032050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.032330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.032634 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.033415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:02.033685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.035709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.044932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 366/730] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:02.045879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.048990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.050027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.050533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.050839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.052339 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:02.052703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.054197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.067790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 365/731] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:02.068697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.129972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.157651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bd9d1-efa5-41b3-be6a-5178b5e5c9ef None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.188214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d433f6d-5509-4ad3-86d8-75aa43742156 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.196160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.196951 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.197240 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.197516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.200306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.206048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.208043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.225214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:02.226003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:02.226950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:02.227888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 367/732] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:02.229098 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.229946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.230553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.231035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.244875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.246764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:02.265908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:02.266859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:02.267749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:02.267749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:02.267749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:02.267749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:02.278668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 366/733] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:02.278668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.343981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab971687-f2af-43c2-8edc-09437c82cc06 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.369394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.389217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.389217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.389217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.389217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.389679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:02.389679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.389679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.396613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 368/734] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:02.397440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.413705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a44e0-2b72-4158-bede-bb8f8ae468d4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.457743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.459435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.460177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.463339 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.464704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:02.465236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.466932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.489202 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 367/735] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:02.501199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.522536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1230b1-b3f3-4ee2-a4d5-ec52d45a9dee None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.560826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.585343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.586064 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.586342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.586619 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.587394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:02.587687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.597854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.609672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 369/736] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:02.619776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.686589 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.721338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1685619-84f4-4f45-9143-78c1c73f089f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.760881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.761681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.761969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.762241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.828782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369edd7-1555-4e7b-a3d9-cab9fb4875b0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:02.839839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.841320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.897881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:02.906052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:02.906894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:02.907177 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:02.907443 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:02.919936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:02.929536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:02.949564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 368/737] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:02.965236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:02.966422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:02.967378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:02.969225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 370/738] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:02.969368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:02.976335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.110365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dadca31-ab57-482d-a9a0-b1ea82309b34 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.128768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.163899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c60a-b3ce-4068-9fb2-5bc27d85ac9b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.188999 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.199711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.200427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.200713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.200986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.201847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:03.202159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.203662 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.209692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.210512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.210787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.211053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.238147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.239478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.251237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 371/739] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:03.256790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:03.267942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:03.269510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:03.270403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:03.270403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:03.270403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:03.270403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:03.279172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 369/740] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:03.286037 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.430023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c1724-9fb9-4c08-9c60-506ca5bb10ae None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.449336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c76aa-b0ce-4b8a-afe8-e23009cb1836 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.481874 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.483096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.483676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.483959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.485953 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:03.487091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.489682 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c3bc4-de9f-4b6c-91db-ec15e8c1c6f2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.493365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.494020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.494295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.494559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.504626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 370/741] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:03.506550 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd094-5c5c-4e70-a6ba-ecd1e97bbc35 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.506841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd094-5c5c-4e70-a6ba-ecd1e97bbc35 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.507119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bd094-5c5c-4e70-a6ba-ecd1e97bbc35 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.509177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 371/742] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:03.510332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd094-5c5c-4e70-a6ba-ecd1e97bbc35 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.536601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.569966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.584794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.586232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c0d1-86e5-407a-a227-43cfce859e9a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.670729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bd094-5c5c-4e70-a6ba-ecd1e97bbc35 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.711232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.712042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.712320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.712623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.720262 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.721750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.726645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 372/743] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:03 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:43:03.726645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.726645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.726645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:03.740973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:03.742391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:03.744520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:03.747218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 372/744] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:03.748913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.756891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:03.758912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:03.837783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.856615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 373/745] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:43:03.858010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:03.875194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:03.884848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8f1e1d-11d8-400b-afff-0d2bbb561c87 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:03.934690 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:03.936347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:03.936843 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:03.937549 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:03.959786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:03.961356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:03.980028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:03.980928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:03.981824 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:03.981824 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:03.981824 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:03.981824 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:03.985804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 373/746] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:03.986631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.001303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83899fbb-572c-4972-a650-ae40c14b1d06 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.048427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.054239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.054239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.054239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.055154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:04.055154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.056970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.070385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 374/747] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:04.071316 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.116318 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de96ba2-4df5-4cc6-9187-65017ae8f48d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.131549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:04.156003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.156772 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.157056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.157329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.158059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.160192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e1045-aa4c-41f3-b426-888cdcd07e5c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.176899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 374/748] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:03 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:04.178036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4fa8-2379-41b9-936d-20a99213cb68 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.178334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4fa8-2379-41b9-936d-20a99213cb68 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.178612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7b4fa8-2379-41b9-936d-20a99213cb68 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.180000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 375/749] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:04.184867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4fa8-2379-41b9-936d-20a99213cb68 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.195786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:04.207116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edabda6-b610-4414-afc2-71393149e4db None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.268170 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.269328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.270010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.270474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.272106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:04.272717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.274573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bab085-0629-440c-a690-1c0bc01375f5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.293628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 375/750] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:04.295608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.296050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.296669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.317769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:04.319325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:04.328842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7b4fa8-2379-41b9-936d-20a99213cb68 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.377822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.379011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.379298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.379706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.381343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=41323e4ae8c145cd950933b4a1733063) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:04.381669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.383123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.390818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 376/751] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/role_assignments?user.id=41323e4ae8c145cd950933b4a1733063&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:04.392212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.420438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 376/752] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:04.438887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.497920 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:04.509754 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:04.545531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e868298-1da4-4706-a3fc-38c3d96c4fdb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.597474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd9ea2-a41f-4958-b3d6-5457b373a6ae None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.602237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.603066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.603351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.603628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.611641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.614525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:04.635567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:04.637785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:04.639578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 377/753] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:04.650729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.681876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.682746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.683027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.683309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.683995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.685500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:04.709012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:04.710010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:04.712008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 377/754] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:04.716531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:04.929959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27246d01-3028-44f7-83f0-4bcde3f2b0ed None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.933393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:04.977435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:04.977920 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128684b-b552-40c0-9609-eb71bf3beaa4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:04.978463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.978896 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.979344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.981175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:04.981669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:04.983216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee6a1e-6f30-4fb7-935b-e81aa020394a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:04.990429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 378/755] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:04.992066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1501-21aa-45f7-b6f9-8770da967e78 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:04.992534 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1501-21aa-45f7-b6f9-8770da967e78 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:04.993211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1501-21aa-45f7-b6f9-8770da967e78 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:04.995012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 379/756] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:04.999881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1501-21aa-45f7-b6f9-8770da967e78 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.021755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:05.029307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.030000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.030282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.030557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.031556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:05.031897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.033371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d714b-79d3-4c7b-94a5-33e6817e8f04 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.049372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 378/757] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:05.051028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.051319 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.051606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.070316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:05.071471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:05.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1501-21aa-45f7-b6f9-8770da967e78 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.171075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 379/758] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:05.172394 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.185957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.186549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.186944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.187147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.187946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:05.188304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.189781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.197643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 380/759] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:43:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:05.203503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.239863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:05.288337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdd1e7-6c38-4b6e-806e-e80c1431b82d None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.312524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:05.329411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.330448 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.330967 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.332458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.333532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=433e683c78a242a695acebce7545f8cf) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:05.333643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c8df8-1ae4-4803-ae8f-daffd76e2b19 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.334187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.335850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.342614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 380/760] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/role_assignments?user.id=433e683c78a242a695acebce7545f8cf&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:05.343724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.382082 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.382800 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.383281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.383391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.384203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] RBAC: Adding query filter params (interface=public, service_id=5267a16ae4a648b7a3b9429413c68176, region_id=RegionOne) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:05.384521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.386028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.395787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 381/761] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/endpoints?service_id=5267a16ae4a648b7a3b9429413c68176&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:05.401487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.470621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cee05c-29aa-4119-a7d9-4d7a2782cdc0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.544932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.545750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.546033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.546317 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.572222 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:05.578303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:05.579278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:05.580200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:05.584521 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 381/762] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:05.586719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:05.605231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26918d75-2fc0-4b2c-8545-a7f0e8dd2e0b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.618611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:05.673532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:05.679226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.680746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.681018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.681291 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.681990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.683412 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.697417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 382/763] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/services => generated 772 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:05.782084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc6f0a-57da-4de5-8937-c3a24aa3ce64 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:05.834370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:05.835840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:05.836580 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:05.837289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:05.838292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:05.838817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:05.840476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:05.857235 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 382/764] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:05.864926 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:06.012301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ffbcc-56f3-4680-82da-13e612b09035 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:06.053386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:06.054172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:06.054461 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:06.054733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:06.067464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:06.068985 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:06.090672 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:06.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:06.092558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:06.092558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:06.092558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:06.092558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:06.093444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 383/765] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:06.099349 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:06.223528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775b2c9-a274-4bd1-8967-e181a90e4679 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:06.230983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:06.275081 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:06.275756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:06.276040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:06.276313 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:06.277079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:06.277385 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:06.278756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:06.305406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 383/766] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:06.310639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:06.330889 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:06.478053 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ffb0-55e7-4005-9101-21085f166d0f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:06.532676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:06.533475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:06.533753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:06.534026 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:06.546867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:06.548340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:06.569563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:06.570527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:06.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 384/767] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:43:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:06.577524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:06.765564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d1a3-39ce-4de2-a1e8-30c7208fa3d9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:06.820123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:06.820758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:06.821035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:06.821308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:06.822232 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:06.822539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:06.824031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e904f-dd2b-40c1-ada6-89a4b4129607 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:06.849114 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 384/768] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:06.854568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:06.958042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.031646 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db040ace-12fb-47a0-b789-699b70a47db7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.078686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b95e8-514e-4fa9-8b4d-c9ea4e037071 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.078863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b95e8-514e-4fa9-8b4d-c9ea4e037071 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.079140 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b95e8-514e-4fa9-8b4d-c9ea4e037071 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.082159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 385/769] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:07.086701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.087010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.087256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.093116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.093913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.094188 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.094456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.115183 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:07.116458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:07.127299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:07.137462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:07.151702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:07.152692 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:07.153549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:07.153549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:07.153549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:07.153549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:07.157613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 385/770] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:07.158869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.173568 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.274182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 386/771] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:07.305047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813661c2-8525-463e-8890-48b5980b6014 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.357108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.369474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.370197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.370482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.370757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.371480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:07.371788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:07.373216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6433e-c8bc-45c6-8071-b8730765321b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:07.387422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 386/772] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:07.389190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b19487-d64d-42ac-8a1a-ae0c16e91470 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.389818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b19487-d64d-42ac-8a1a-ae0c16e91470 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.390243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b19487-d64d-42ac-8a1a-ae0c16e91470 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.398784 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 387/773] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:07.400202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b19487-d64d-42ac-8a1a-ae0c16e91470 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.454219 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff70ce0-54a0-4009-85db-79a3987a1b11 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.505351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.506736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.507139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.507421 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.521282 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:07.527630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b19487-d64d-42ac-8a1a-ae0c16e91470 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2cb7926a-59b7-460d-8904-7ab59d4c1f5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:07.531077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:07.532034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:07.533808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 387/774] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:07.535044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.535977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.535977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.557780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:07.559439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:07.571254 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.578396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.579601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.579883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.580157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.589201 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:07.590873 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:07.610063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:07.611093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:07.611977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 388/775] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:07.613002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.650911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 388/776] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:07.652122 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.687883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.769235 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b23fa5-550e-4f88-8578-000312f9ef9e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.787891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8a5cf-9a94-40af-995e-68476b3fdd57 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.813507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.814204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.814476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.814768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.816935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:07.817288 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:07.818722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:07.831572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 389/777] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:07.832825 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.839879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:07.840963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:07.841482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:07.842040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:07.843203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:07.843727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:07.845707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:07.859570 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 389/778] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:07.860822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:07.866833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.957850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91841f-f3a5-4f5f-af0a-822ed210b873 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:07.976658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:07.995421 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df21467-7041-48d2-a42d-1da243290dd8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.011451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.012224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.012541 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.012820 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] PATH_INFO: `/v3/services/network` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.013483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.014855 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:08.029376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:08.030517 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:08.031356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 390/779] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:08.034516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.043402 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.044201 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.044513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.045049 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.057568 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.059014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7620aab6-cb4b-490b-89fb-85c421a64cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:08.078984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:08.080528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 390/780] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:08.082291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea70e-b1f2-4091-95e4-c1c6f37b6131 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.082665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea70e-b1f2-4091-95e4-c1c6f37b6131 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.082665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dea70e-b1f2-4091-95e4-c1c6f37b6131 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.084363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 391/781] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:08.089192 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dea70e-b1f2-4091-95e4-c1c6f37b6131 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.155246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c60e95-7208-4b01-b209-e16fccf9ca81 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.155767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.196746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.197854 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.198040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.198315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.200528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=6196d18cd1b44984a2b1c0602745314f, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.201989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.209962 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dea70e-b1f2-4091-95e4-c1c6f37b6131 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.224440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 391/782] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:08 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:43:08.224440 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.266066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.267249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.267879 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.268168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.269614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.271136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] RBAC: Adding query filter params (name=network) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:08.271519 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.273698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc3d76-7e11-41eb-ab5c-55dba94fc6d6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.296717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 392/783] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:08.298268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.299206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.299485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.320719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:08.322063 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:08.389347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5818d6-c2ea-4cdc-84df-b35cfd9fcbce None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.411088 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 393/784] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:08.412642 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.439281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.440060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.441078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.441362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.442131 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:08.442445 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.445058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.463184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 392/785] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:08.464176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.479491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.561936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd70de-0431-4f52-8375-eb3486432b9a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.585720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.597984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a261089-8348-4425-85f6-485bb6e3c456 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.623476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.624487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.624774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.625052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.625946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:08.626258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.628621 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.640539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 394/786] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:08.642035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.655526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.656331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.656638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.656909 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.669519 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:08.678037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:08.679063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:08.679948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 393/787] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:08.680731 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.768068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cb867-3573-484c-bb67-c6bdbe09e6f5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.785285 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.796535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff85b9-d90e-4f74-b862-19f28e32c66c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.812053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.813252 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.813677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.813952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.822182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.823625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.838526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.839540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.840017 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:08.840178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:08.841120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:08.842048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 395/788] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:08.842716 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.843647 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.844483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] RBAC: Adding query filter params (interface=public, service_id=13d460c728014501a855c033dd11f878, region_id=RegionOne) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:08.844802 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:08.846205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99678ec-784c-4467-b875-ffed3ee18f9f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:08.852531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 394/789] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/endpoints?service_id=13d460c728014501a855c033dd11f878&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:08.853870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bea5-3cbc-4de5-bf07-1b6977807b62 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.854156 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bea5-3cbc-4de5-bf07-1b6977807b62 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.854428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71bea5-3cbc-4de5-bf07-1b6977807b62 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.856709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 395/790] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:08.858054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bea5-3cbc-4de5-bf07-1b6977807b62 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:08.877327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:08.957600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e61a5d-aa2e-47c0-b33c-07787125f1b5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.981656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71bea5-3cbc-4de5-bf07-1b6977807b62 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:08.997488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:08.998136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:08.998419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:08.998692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:08.999426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:08.999731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.001155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.015026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 396/791] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.015026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.026686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.027362 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.027630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.027893 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.028760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:09.029069 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.030600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0a828-baa0-4016-9b8a-529b8b7899d0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.058537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 396/792] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.058537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.058537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.058537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.068304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.075804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:09.077047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:09.141076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03b900-f3f1-46c5-a5d4-41f2ddaebf67 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.160425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 397/793] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:09.160425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.163609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.181448 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.182102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.182382 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.182657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.183336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.184708 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.190098 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 397/794] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.190880 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.302257 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a71d92-ef1a-4957-9504-0d68cc37ada3 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.309432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.313714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d456b-3da3-4f80-9ba8-a26434c89e0a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.346838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.348017 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.348482 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.348919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.355686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.356712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.357099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.357481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.362855 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.364897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.371177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.372839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:09.386785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:09.387914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:09.388817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:09.388817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:09.388817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:09.388817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:09.390688 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 398/795] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:09.391683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:09.394368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:09.395237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:09.396093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:09.396093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:09.396093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:09.396093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:09.400161 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 398/796] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:09.401089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.455472 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.509943 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66787a3-3331-4e2d-aa59-b79cee0fa6f5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.534908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683c250-0866-49b9-b168-c5356a923e6e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.559337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.560435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.560726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.560993 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.561642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.563016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:09.579936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:09.581138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:09.582174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 399/797] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:09.583180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.585778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.586524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.586804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.587074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.587864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:09.588172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.589622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.608748 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 399/798] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.609613 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.627563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.714518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a5239-8872-47de-a7a4-1834b0323f44 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.733145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e1b80-b062-44a6-b67f-0c8de88087d1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.733762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.762428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.763881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.764662 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.765149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.766785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:09.767312 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.771105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.779956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.781483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.781767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.782087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.782808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:09.783115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.785532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.787892 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 400/799] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.788845 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.796847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 400/800] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:09.797809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:09.905935 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:09.919837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727e591-c408-4ca8-ae5e-a5f4d0323d61 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.939812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e1913-146f-4e31-9b5e-471c1eb3dbae None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:09.957650 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.958476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.958759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.959036 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.969558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:09.971291 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:09.986603 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:09.987460 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:09.987738 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:09.988010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:09.991127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:09.992039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:09.993561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:09.996660 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.000582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.000828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 401/801] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:10.002033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:10.014799 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:10.015779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:10.016717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:10.023156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 401/802] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:10.023156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.065532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:10.143084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ec66-cfa2-4085-96b4-bda1722f52f2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.170503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50357c74-6a29-4cdd-9d8b-220fafb56165 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.191503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.193142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.193653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.194384 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.195582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:10.196545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.198668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.211101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 402/803] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:43:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:10.211101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.213691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.214373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.214651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.214923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.215822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:10.216129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.218478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.222066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:10.244670 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 402/804] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:43:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:10.245619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.348385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ded3d-ea1f-4ffd-91ff-8c9e9ebbb59a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.350223 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:10.384070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b433059-ebf2-4bbf-8c2a-62a8bc4f5b11 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.394653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.396105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.396641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.397526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.398804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:10.399335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.402011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.426586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 403/805] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:10.428168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.431361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.432043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.432328 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.432636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.433393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.435934 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.469829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 403/806] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:10.469829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.587704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:10.654491 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ed187-a3b1-463c-882e-061df86c7723 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.724414 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.725179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.725551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.725736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.727518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc87e-0ea5-4971-ad63-c71c56cfe1a0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:10.739842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.741471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:10.761936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:10.762901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:10.763773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:10.763773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:10.763773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:10.763773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:10.768484 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 404/807] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:10.770537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:10.773650 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:10.774560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.774853 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.775116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/41323e4ae8c145cd950933b4a1733063/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.786816 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:10.805746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=41323e4ae8c145cd950933b4a1733063, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:10.807184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a479c9dd-4d54-4c05-a7e9-439c3c26f34f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:10.824589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 404/808] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:10 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/41323e4ae8c145cd950933b4a1733063/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:43:10.826370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8bcb3-94e6-49a1-969c-0074a87c67a9 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.826552 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8bcb3-94e6-49a1-969c-0074a87c67a9 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.826827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8bcb3-94e6-49a1-969c-0074a87c67a9 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.828381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 405/809] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:10.838302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:10.838642 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:10.838918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:10.866352 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:10.868647 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:10.979968 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372ebe-5013-4d33-b31b-21252c9b6aad None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.029128 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 406/810] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:11.043462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.044195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.044499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.044767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.045594 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:11.045907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.047318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.059755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.069123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 405/811] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:11.076000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.099412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.197615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.243542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bec09-9d62-4792-a23e-731f33e038e6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.260717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa0d91-784e-4a85-b8eb-739b4851cac0 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.301789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.302921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.303412 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.303877 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.312145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.314067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.319988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.320904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.321227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.321515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:11.332879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:11.333794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:11.334669 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:11.341516 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:11.341804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 406/812] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:11.343173 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:11.347209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:11.348130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:11.349227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 407/813] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:11.354506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.478233 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.484708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267801a4-ed9f-4c47-95c5-bec02f01b88f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.512381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1c2aa-9409-42a2-b688-f7182f617e54 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.518900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.528949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26243403-7a38-4e9d-9606-74727921536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.529906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26243403-7a38-4e9d-9606-74727921536a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.530336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26243403-7a38-4e9d-9606-74727921536a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.530770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26243403-7a38-4e9d-9606-74727921536a None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.531716 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26243403-7a38-4e9d-9606-74727921536a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:11.532196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26243403-7a38-4e9d-9606-74727921536a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.535870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26243403-7a38-4e9d-9606-74727921536a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.562436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 407/814] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:11.569285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26243403-7a38-4e9d-9606-74727921536a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.588551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.592069 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.592565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.592995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.594252 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:11.594570 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.595970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31217ba-c2cb-4819-a0eb-cf015ae4bca5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.610683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 408/815] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:11.612736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554b97-6bbb-4cbf-b298-481fa702d9de None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.613026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554b97-6bbb-4cbf-b298-481fa702d9de None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.613304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09554b97-6bbb-4cbf-b298-481fa702d9de None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.619882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 409/816] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:11.619882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554b97-6bbb-4cbf-b298-481fa702d9de None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.713317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26243403-7a38-4e9d-9606-74727921536a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.751852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.753704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.754206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.754679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/433e683c78a242a695acebce7545f8cf/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.756959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=433e683c78a242a695acebce7545f8cf, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.758628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c89d25-0239-4de6-8d4d-d452ca0cd758 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.763508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09554b97-6bbb-4cbf-b298-481fa702d9de None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:11.774154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 408/817] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:11 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/433e683c78a242a695acebce7545f8cf/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:43:11.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.776009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.776448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.811389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:11.813914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:11.852682 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:11.853584 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:11.853867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:11.854138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:11.885275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:11.888197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:11.914503 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.928491 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:11.932187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:11.933237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:11.934135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:11.934135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:11.934135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:11.934135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:11.943213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 410/818] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:11.948993 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:11.957033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 409/819] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:12.004103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.087601 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e364389-9a40-48c4-ac7c-fedce7d985d3 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.129056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07ad66-e7db-4d12-bd68-2c3697a44a4a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.129884 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.130845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.131129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.132337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.133114 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:12.133423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.134858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.160080 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 411/820] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:12.165844 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.174028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.174946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.175232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.175505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] PATH_INFO: `/v3/services/network` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.176170 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.177792 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:12.191324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:12.193957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:12.206447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 410/821] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:12.207246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.254948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:12.257847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:12.325089 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44206e16-f43d-40e8-8f59-bc15e9739636 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.325315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbfdf8-6d1d-4ca6-9f61-ac972ba77865 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.366675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.367510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.367786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.368063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.368814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:12.369117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.370502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.380313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:12.389964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.425115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.426018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.426291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.426556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.443260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.445380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:12.475587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:12.477984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:12.480351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:12.487610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 412/823] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:12.493944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.633180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e49b8-dd85-439c-adb0-9e254a7ff1d5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.662307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:12.701191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.702174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.702559 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.702938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.703745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] RBAC: Adding query filter params (type=network) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:12.704155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.706579 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.712411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 412/824] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:12.718223 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.726779 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b8a50-1a2f-4669-88fa-6e8e5f96ccc9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.766902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:12.802119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.802809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.803096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.803368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.804217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:12.804557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.805991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.833664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 413/825] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:12.845526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:12.847490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956f417-7810-4fa8-b291-1c1444a72cf9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:12.896385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:12.897084 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:12.897370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:12.897647 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:12.898406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:12.899795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:12.943849 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 413/826] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:13.061732 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac22fc6-b538-4cc2-bba1-80015fa052a8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:13.115677 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:13.116338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.116645 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.116911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.167618 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:13.197031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:13.199170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc254aa2-80c2-4c02-adac-43de6ce61477 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:13.319105 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 414/827] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:13.324321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:13.363704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:13.453212 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3dce9-763c-43da-abc9-d55df7b76ce1 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:13.505283 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:13.506163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.506474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.506749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.513680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9183-590f-4295-bb2d-ba0c7b2b4d5b None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.514042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9183-590f-4295-bb2d-ba0c7b2b4d5b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.514244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9183-590f-4295-bb2d-ba0c7b2b4d5b None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.515844 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 415/828] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:13.520186 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:13.521797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:13.522573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.523181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.523457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:13.546489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:13.547421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:13.548307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:13.548307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:13.548307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:13.548307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:13.550703 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 414/829] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:13.558269 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:13.562586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:13.564548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:13.647709 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 416/830] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:13.670314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:13.676169 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:13.697720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04771c-3181-422c-85c7-2262aff298b0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:13.699357 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:13.751260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:13.751844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.752127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.753146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.754459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:13.754720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:13.756944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1092ae2-2d1e-4e8d-af86-09e7da2d843a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:13.776390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 415/831] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:13.777839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47cd10-2dc9-488f-bd9a-bb6c024a4c28 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.778133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47cd10-2dc9-488f-bd9a-bb6c024a4c28 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.778411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47cd10-2dc9-488f-bd9a-bb6c024a4c28 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.783502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 416/832] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:13.783502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47cd10-2dc9-488f-bd9a-bb6c024a4c28 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:13.794726 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02558e-c788-48a3-add5-554b4f749a66 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:13.833958 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:13.834762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.835047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.835327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.835991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:13.837476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c6863841-bcc9-48c9-ad6a-d7aa024c8166 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:13.849836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:13.851676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:13.853329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 417/833] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:13.854152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.854152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.854152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.877101 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:13.879270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:13.922521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47cd10-2dc9-488f-bd9a-bb6c024a4c28 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:13.950167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:13.957528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 418/834] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:13.958512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:13.977637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:13.978570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:13.978865 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:13.979140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:13.990092 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:13.991570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:13.995246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:14.008363 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:14.009483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 417/835] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:14.010444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.089298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c633b-df41-4c74-b9cc-6c2c66606844 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.125374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.126056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.126345 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.126627 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.127929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:14.128534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.130272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.131865 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec722d7-981a-4bf1-83b2-705fb7d5da9a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.135761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 419/836] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:14.136562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.184338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.185603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.186336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.186798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.200822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:14.210823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:14.211759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:43:14.212643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:14.218516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 418/837] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:14.218516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.240583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.283625 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.286097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecca680-4cce-468b-8ca6-df1fc24172ba None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.322973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.323647 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.323926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.324196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.325045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:14.325355 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.326841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.333299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d5da-f9f5-4fa6-adcb-7dc9bcefc22e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.346159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 420/838] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:14.346159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.377255 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.378658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.378933 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.379206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.379908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:14.380207 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.381614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.386831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 419/839] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:14.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.496310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96d60-e490-4fd5-a013-57a0725910ad None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.532632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.532946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e63960-e255-470a-ad16-0fd3373db986 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.548510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.549119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.550023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.550299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.551070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:14.551376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.552798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f9b34-f312-4cbf-b83f-e8f24f26b123 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.564458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 421/840] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:14.564458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b00d-7ab4-4ef8-b12c-56af53320a24 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.564458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b00d-7ab4-4ef8-b12c-56af53320a24 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.564775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63b00d-7ab4-4ef8-b12c-56af53320a24 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.566838 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 422/841] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:14.567007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.571391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63b00d-7ab4-4ef8-b12c-56af53320a24 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.580196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.581689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.581972 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.582245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.583712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:14.584033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.585510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa9ed8-a0aa-4a8e-8d97-7b47cfe63e6c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.596835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 420/842] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:14.596835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.596835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.596835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.619116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:14.620527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:14.724904 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 421/843] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:14.726194 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63b00d-7ab4-4ef8-b12c-56af53320a24 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.748598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.776811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.777617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.777897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.778167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.794048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:14.795523 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:14.816982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:14.817905 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:14.818802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:14.818802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:14.818802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:14.818802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:14.818802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 423/844] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:14.823728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:14.870247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.876997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e3065-4320-4971-9cf5-737c86623997 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.897499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:14.929203 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:14.930024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:14.930304 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:14.930573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:14.946385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:14.954821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:14.956533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:14.957535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:14.959909 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabaff3-8f73-48e9-bebe-c87d85af5e21 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:14.960629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 422/845] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:14.969385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.004492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.005555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.006033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.006529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.007500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:15.008020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.009648 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.025640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 424/846] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:15.030806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.104992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0e1e-34a3-49dd-9333-9f4df98beb6f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.145469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.146612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.147130 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.147651 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.148812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:15.149321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.150937 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.152745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.160478 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf45f3-b4af-40a0-9f7b-1ae280ceee46 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.164254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 423/847] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:15.169573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.203059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.204108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.204655 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.205103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.213735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.215500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.226526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:15.236624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:15.238167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:15.239196 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:15.242371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 425/848] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:15.250143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.312774 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80576f3-5a1e-4124-b95d-b94eaf79980c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.359999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.360838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.361060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.361333 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.373548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a4aa-0520-4a6e-b6bd-778547a7882a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.379062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:15.404339 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:15.405331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:15.406225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:15.406225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:15.406225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:15.406225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:15.406225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 424/849] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:15.411682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.419364 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.420333 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.420924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.421418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.422598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:15.423164 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.425530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.447400 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 426/850] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:15.456432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.476827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.547740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.559181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e085a-2082-4abe-827b-3e3be7a9795c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.604931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2631a-1dfb-4176-94ef-058520be8a26 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.609980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.610708 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.610938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.611216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.612003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:15.612313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.613730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.639460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 425/851] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:15.640503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.661830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.662952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.663459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.663936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.772713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e44d8-f9a8-40b9-820f-7c44dce4d3dd None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:15.788079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.790177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.813776 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.817483 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:15.818511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:15.818940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:15.819361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] PATH_INFO: `/v3/users/swift` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:15.829145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:15.831603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:15.849163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:15.850178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:43:15.851109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:15.853465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 426/852] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:15.859247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:15.879563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:15.932713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 427/853] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:15.934788 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.009759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529b952-1dcb-4978-8880-3de76f1a4bde None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.064107 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.066666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.067210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.067637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.068665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:16.069116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c591e2-1de3-45b1-902e-ce2a69a492e0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.069397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.071300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.087456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.091889 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 427/854] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:16.098881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.115619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.116812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.117550 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.118030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.131767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.134877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:16.155016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:16.155938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:16.156866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:16.156866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:16.156866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:16.156866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:16.159882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 428/855] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:16.168943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.189139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.232677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87e93e-0f50-4b1d-a119-d833e6a2dd40 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.279331 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.280384 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.281030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.281485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.320149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340771a4-2657-4bb4-a108-3c17572f9488 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.357588 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.358634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.359141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.359616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.360577 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:16.361216 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.363501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.377881 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.378553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 429/856] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:16.382108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.383773 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.430288 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.471984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.514001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 428/857] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:16.520963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.542429 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac294ae-5ea6-43f3-9a48-d726e845c464 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.593574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.594711 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.595000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.595278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.606193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.609112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:16.636977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:16.637965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 430/858] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:16.638866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.668033 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b38ba-d44b-4a58-a312-72a7e796e08b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.715956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.716790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.717123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.717403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.731424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.733340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.735707 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.759439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1828-cde1-49f9-85a0-5ddeebbac467 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:16.759665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:16.760581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:16.761568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:16.761568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:16.761568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:16.761568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:16.764542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 429/859] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:16.768736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.802241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:16.804310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.805034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.805321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.805687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.806481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:16.806806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.808229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a23e16-dddd-49e3-8c3a-4ed6ab96060a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.821540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 431/860] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:16.823692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5a0d8-14ba-4f1a-ae1d-0454d3f110e4 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.824173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5a0d8-14ba-4f1a-ae1d-0454d3f110e4 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.824700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5a0d8-14ba-4f1a-ae1d-0454d3f110e4 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.830558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 432/861] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:16.830558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5a0d8-14ba-4f1a-ae1d-0454d3f110e4 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:16.916701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c151-b4fb-442b-9770-95a32da25d9e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.956177 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:16.956920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.957044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5a0d8-14ba-4f1a-ae1d-0454d3f110e4 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:16.957259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.957532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.958259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:16.958590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:16.960045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361496-8f38-46aa-ad00-2e018817736e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:16.976039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 430/862] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:16.977952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dce3f-4656-49c3-a2a6-4ce9c661faa5 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:16.978737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dce3f-4656-49c3-a2a6-4ce9c661faa5 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:16.979779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dce3f-4656-49c3-a2a6-4ce9c661faa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:16.997795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-284dce3f-4656-49c3-a2a6-4ce9c661faa5 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:16.999099 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-284dce3f-4656-49c3-a2a6-4ce9c661faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:17.001749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.002940 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.003435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.003868 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.004782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=41323e4ae8c145cd950933b4a1733063) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:17.005099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.006531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.013309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 433/863] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:16 2022] GET /identity/v3/role_assignments?user.id=41323e4ae8c145cd950933b4a1733063&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:17.014180 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.031636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.088806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 431/864] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:17.090600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f145e-3eb1-4f51-894a-7f5ba03c1901 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.091163 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f145e-3eb1-4f51-894a-7f5ba03c1901 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.091648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f145e-3eb1-4f51-894a-7f5ba03c1901 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.093592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 432/865] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:17.100449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.100449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.100449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.124808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:17.126047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:17.136759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.162719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8858d-d04d-42bb-bf16-48ca23e90222 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.212943 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.213694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 433/866] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:17.213817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.214004 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.214224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.222467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.222646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.224213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:17.242860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:17.243916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:17.244987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 434/867] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:17.245550 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.338551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.346783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466f11b-1b27-4d03-aa97-92e011d4d2f9 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.373101 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9312-44ed-406d-84c0-c34064f2c937 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.390806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.391500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.391775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.392025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.393292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.394778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.415120 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 434/868] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:17 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:17.416017 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.419423 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.438122 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.439437 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.439988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.440525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.441451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.443159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:17.457877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:17.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:17.459704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:17.463070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 435/869] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:17.469257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.546129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4a9b-863b-4a23-83f3-ae8cdf9b46f1 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.587734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.588485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.588775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.589106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.589926 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:17.590246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.591755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.611401 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 435/870] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:17.613581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.614534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12938bf-abd2-4be9-bd82-297e01302a54 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.659351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.669693 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.670373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.670658 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.670935 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.671657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:17.671963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.674440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.681134 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 436/871] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:17.689434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:17.727076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:17.746674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ec8a1-1bbe-4784-8d1a-87a223d99bd3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.791149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.791921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.792268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.792724 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.793720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=433e683c78a242a695acebce7545f8cf) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:17.794101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.795601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7727f5-0591-417b-9247-58be63aeeb1b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.803410 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 436/872] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/role_assignments?user.id=433e683c78a242a695acebce7545f8cf&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:17.832606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766728-5d01-4722-bcec-b7c1ae313b34 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:17.875498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:17.876188 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:17.876508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:17.876797 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:17.878489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:17.878798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:17.880265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c0757-fa22-4816-91d4-f0fcbd52af4e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:17.886372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 437/873] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:19.515503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0e1f-fc4f-43fc-bb64-1b982d80153f None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:19.516058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0e1f-fc4f-43fc-bb64-1b982d80153f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:19.516058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc0e1f-fc4f-43fc-bb64-1b982d80153f None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:19.518094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 437/874] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:19.523493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922361fc-366b-4c80-886e-3d586f501537 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:19.523913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922361fc-366b-4c80-886e-3d586f501537 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:19.524084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922361fc-366b-4c80-886e-3d586f501537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:19.541912 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-922361fc-366b-4c80-886e-3d586f501537 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:19.543133 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-922361fc-366b-4c80-886e-3d586f501537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:19.605890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:19.625913 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 438/875] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:19.671676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0e1f-fc4f-43fc-bb64-1b982d80153f None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:19.799969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084362a-9f0c-432e-ae8e-c9e472dcc2d1 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:19.800461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084362a-9f0c-432e-ae8e-c9e472dcc2d1 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:19.801507 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084362a-9f0c-432e-ae8e-c9e472dcc2d1 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:19.804876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 439/876] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:19.813139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:19.813787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:19.814080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc0e1f-fc4f-43fc-bb64-1b982d80153f None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:19.814287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:19.843354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:19.845188 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:19.846326 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:19.858054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:19.858900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:19.859183 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:19.859455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:19.873620 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:19.883224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:19.884155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:19.885009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 438/877] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:19.892224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:19.952493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 440/878] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:19.974738 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.032436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aff0b2-883d-40e4-8287-3967af28e2dc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.072838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.073842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.074395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.074864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.075808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:20.076585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.078328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.088762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 439/879] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:20.094799 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.107975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90807cb5-4196-43b4-a18d-3ffc1ac08962 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.131954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.152778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.154117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.154593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.155072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.156032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.157829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.159521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2f9462a3-2866-41f3-bac6-a12e0535cb87 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:20.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:20.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:20.175593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:20.180444 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 441/880] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:20.181989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5242-2603-4870-bd46-1cd27be3227a None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.182283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5242-2603-4870-bd46-1cd27be3227a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.182557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f5242-2603-4870-bd46-1cd27be3227a None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.188534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 442/881] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:20.188534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f5242-2603-4870-bd46-1cd27be3227a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.233830 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd27e9-04b9-4b5e-bbee-ab8fac4383e0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.270767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.271591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.271879 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.272155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.284911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.286417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5e80b2dd-6a00-460e-b7f7-903298ba92fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:20.304431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:20.305377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:20.306268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:20.306268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:20.306268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:20.306268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:20.306268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 440/882] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:20.306981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dcfdc-d219-4d92-8f67-3c728b376784 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.307080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dcfdc-d219-4d92-8f67-3c728b376784 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.307353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dcfdc-d219-4d92-8f67-3c728b376784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.313697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f5242-2603-4870-bd46-1cd27be3227a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.334232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8e9dcfdc-d219-4d92-8f67-3c728b376784 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:20.335360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8e9dcfdc-d219-4d92-8f67-3c728b376784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:20.359614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.360305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.360615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.360893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.362200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:20.362809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.364151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.369599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 443/883] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:20.370734 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.408769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 441/884] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:20.410111 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0854ea-f121-48a4-83a8-cb80e3286e43 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.410394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0854ea-f121-48a4-83a8-cb80e3286e43 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.410667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0854ea-f121-48a4-83a8-cb80e3286e43 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.411920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 442/885] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:20.412626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0854ea-f121-48a4-83a8-cb80e3286e43 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.424542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.453209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.487216 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16839977-aa98-42e2-8dbf-6d575f954b84 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.523107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.523760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.524325 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.524325 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.525032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:20.525338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.526694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d70c5af-b9a7-45c1-95cc-8cdaf3fb45c3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.530119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0854ea-f121-48a4-83a8-cb80e3286e43 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.540585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 444/886] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:20.541881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.542166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.542436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.558532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:20.559712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:20.573372 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.574035 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.574313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.574591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.575296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:20.575602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.577008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.588153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 443/887] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:20.588977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.635661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 445/888] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:20.636745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.696207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.713652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.722209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20073739-18f0-4b52-b7e8-965ed73235e5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.759134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.759935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.760221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.760516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.761168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.762539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.769599 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883e9d4-00b1-48e5-930b-9034b68ce4bf None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:20.774973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:20.775964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:20.776844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 444/889] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:20.777647 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.811793 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.812624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.812906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.813179 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.820850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.822274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:20.836663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:20.842887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 446/890] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:20.843869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.885093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6f74a-2949-49f7-a6fa-9f64e3e37de1 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.924695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.925354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.925635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.925908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.926695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] RBAC: Adding query filter params (interface=public, service_id=30eafbebfcd5494096b8f3386cde7fdd, region_id=RegionOne) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:20.927005 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.928383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:20.933437 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 445/891] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/endpoints?service_id=30eafbebfcd5494096b8f3386cde7fdd&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:20.934218 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:20.939155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.946177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156049fb-bd37-4507-9461-604925e91d46 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:20.961668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:20.983207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:20.983867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:20.984146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:20.984442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:20.985162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:20.986562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.001574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 447/892] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:20 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:21.003398 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.055663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9438201-3165-4bec-ae69-d6da2d3e7357 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.103344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.104141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.104458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.104749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.105686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:21.105992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.107591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.122083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 446/893] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:21.122083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.143124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5611ca6-74bb-4a7a-aa75-dc1f7a59037c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.192295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.193046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.193329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.193603 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.194488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:21.194801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.196266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.216832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 448/894] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:21.218015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.229831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:21.238592 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:21.255148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69074e9d-dd66-4e7a-9990-ceba95a7a680 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.296667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.297340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.297622 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.297897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.298584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.300549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.306797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 447/895] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:43:21.307623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.358769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0495-19e0-4e71-959c-b7745607db70 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.411931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.412654 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.412941 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.413220 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.413958 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:21.414272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.415706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.428666 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 449/896] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:21.458430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36462c55-1dbb-4d0f-90af-d36576eadeb0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.506521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.507598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.508096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.508613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.527419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:21.597450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.598978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.716356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 448/897] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:21.722564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.822197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:21.856030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa59e9-5667-48b7-9a18-af9df9980ade None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:21.895546 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:21.896360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:21.896676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:21.896950 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:21.909368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:21.911003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:21.929899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:21.930816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:21.931715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:21.931715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:21.931715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:21.931715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:21.935666 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 450/898] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:21.937479 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:21.988938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:22.049153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d86ee-e933-4d98-ad0f-0886c47f6a3f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:22.085789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:22.086448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:22.086726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:22.086995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:22.087714 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:22.088015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:22.089414 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:22.102260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 449/899] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:22.108067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:22.234892 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973e2b7-06cb-4b20-b42b-81e8a97f8418 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:22.275509 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:22.282477 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:22.283615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:22.284119 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:22.284652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:22.292526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:22.294274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:22.309251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:22.310206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:22.311112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:22.319115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 451/900] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:22.320880 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:22.435724 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fadb4ac-fcc7-4283-bcba-e8547d29a36c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:22.443516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:22.473508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:22.474172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:22.474451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:22.474725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:22.475511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:22.475821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:22.477294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9a47a-dd1d-49b9-9159-e40ffc934759 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:22.490010 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 450/901] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:22.496181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:22.610040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37743-b0dd-429a-ab9c-ecd3360cf355 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:22.650604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:22.651638 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:22.652154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:22.652664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:22.653704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] RBAC: Adding query filter params (role.id=73adac171d2d42b28c7c3744379a89e7, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=41323e4ae8c145cd950933b4a1733063) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:22.654244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:22.655882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760fd4e-bbb3-4c7e-94ce-df824b522dbd None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:22.666304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 452/902] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:22 2022] GET /identity/v3/role_assignments?user.id=41323e4ae8c145cd950933b4a1733063&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=73adac171d2d42b28c7c3744379a89e7 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:23.382721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddf19-a8e8-428c-87bf-f5992dbb8baa None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.383210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddf19-a8e8-428c-87bf-f5992dbb8baa None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.383210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ddf19-a8e8-428c-87bf-f5992dbb8baa None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.384798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 451/903] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:23.390254 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.390666 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.390837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.418465 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:23.419664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:23.474434 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:23.494013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 453/904] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:23.526319 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddf19-a8e8-428c-87bf-f5992dbb8baa None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:23.653431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ddf19-a8e8-428c-87bf-f5992dbb8baa None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:23.705620 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:23.706474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.706763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.707042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.707744 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:23.709450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e1a6ce8f-66f2-450a-ac5c-0aaf5e0ff3c1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:23.723656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:23.724707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:23.726195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 452/905] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:43:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:23.730974 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:23.752859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4c7f-80ba-4233-b08f-9fb490322384 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.753166 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4c7f-80ba-4233-b08f-9fb490322384 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.753408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c4c7f-80ba-4233-b08f-9fb490322384 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.754773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 453/906] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:23.760104 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9cfdd-5bdf-4789-86b5-e191cb23ea79 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.760421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9cfdd-5bdf-4789-86b5-e191cb23ea79 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.760718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9cfdd-5bdf-4789-86b5-e191cb23ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.780217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c9e9cfdd-5bdf-4789-86b5-e191cb23ea79 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:23.781465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c9e9cfdd-5bdf-4789-86b5-e191cb23ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:23.808721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:23.866081 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f43ab20-5b28-4aa7-8a63-dd4d9f72b4fa None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:23.877318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 454/907] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:23.880502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505adf-9e65-46d5-94d2-d502c8058e91 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.880794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505adf-9e65-46d5-94d2-d502c8058e91 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.881072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505adf-9e65-46d5-94d2-d502c8058e91 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.886521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 455/908] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:23.888072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.888357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.888656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.907500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:23.909343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:23.912483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:23.913146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:23.913423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:23.913692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:23.914387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:23.914689 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:23.919877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:23.934331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 454/909] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:23.934331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:23.991551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:23.998549 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 456/910] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:24.002349 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c1b0f-0714-45c8-8365-858e6157f7d9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.093350 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.094151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.094430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.094702 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.095995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.097406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:24.111447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:24.112720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:24.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 455/911] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:24.114311 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.117486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:24.129608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b6abc-9b8b-4952-82ad-8784c3f486ff None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.167138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.167720 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.168000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.168267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.169009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:24.169310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.170677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.175663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 457/912] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:24.176379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.245907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da532a7-34e7-4b7e-91d0-2cf0856d4671 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.259649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:24.287694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1cf11-ad76-4416-91b5-e587cf2e4df2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.294418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.295097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.295433 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.296087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.296986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.298643 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.320484 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 456/913] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:24 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:24.320484 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.350835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.351537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.352091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.352851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:24.353167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.354612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.360507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 458/914] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:24.366079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.388912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:24.459585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061ae92-ac40-4632-8e3b-78c6089f94e1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.507859 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.508796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.509163 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.509539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.510465 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] RBAC: Adding query filter params (interface=public, service_id=3602c1725fbe4f1a87cb1ec515a399e2, region_id=RegionOne) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:24.510987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.512685 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.522469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 457/915] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:24 2022] GET /identity/v3/endpoints?service_id=3602c1725fbe4f1a87cb1ec515a399e2&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:24.523071 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cb96-d35d-4a2c-99d9-a58283732d37 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.527700 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.570235 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.571072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.571352 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.571833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:24.572768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.573255 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:24.573564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.575058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.583665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 459/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:43:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:24.589430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:24.651531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f69d-843d-44ae-99ce-e38bd1b34815 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.696956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:24.697156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.698230 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.698512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.698786 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.699930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.701581 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf9064-e74b-4268-bdca-9348cd99b89a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.714031 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 458/917] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:24 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:43:24.732560 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77050d0-82c0-4acf-aa82-a9b2a476f55f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:24.777015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:24.778109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:24.778596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:24.779097 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:24.780087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:24.781859 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2e528-a4f4-4461-8728-0be6a19b98a6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:24.806997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 460/918] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:25.109400 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae602-ad04-4a37-abe9-a20946f76afb None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.109914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae602-ad04-4a37-abe9-a20946f76afb None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.110305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ae602-ad04-4a37-abe9-a20946f76afb None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.112805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 459/919] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:25.118955 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.119381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.119765 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.138305 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:25.139573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:25.221584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 461/920] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:25.245099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ae602-ad04-4a37-abe9-a20946f76afb None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:25.357589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:25.362600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ae602-ad04-4a37-abe9-a20946f76afb None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:25.402152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:25.402965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.403248 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.403574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.414876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:25.422918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:25.424170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:25.425237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 460/921] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:25.433246 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:25.502673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:25.547476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b2ca0-b5b7-416e-919e-80bc19794fb8 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:25.585571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:25.586637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.587157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.587625 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.588648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:25.589155 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:25.590787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:25.600715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 462/922] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:25.608525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:25.727236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89073ecd-591f-4c96-a3c2-12911b087505 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:25.764054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:25.765289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.765914 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.766175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.778685 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:25.780127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:25.802306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:25.804622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:25.807986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:25.807986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:25.807986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:25.807986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:25.807986 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 461/923] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:25.812613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:25.941145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fb260-05b0-49b2-b1ad-68bb20074244 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:25.978728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:25.993062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:25.993799 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:25.994083 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:25.994407 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:25.995159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:25.995474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:25.997127 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.013572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 463/924] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:26.020060 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:26.036252 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:26.136281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c608b1-03ac-490f-91b4-00ce3324ea6c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:26.176111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:26.177039 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.177323 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.177598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.187207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:26.188688 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:26.205015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:26.207187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:26.209536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:26.211419 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 462/925] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:26.212728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:26.331575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e269b-0426-44d7-ba9e-6ad037621be5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:26.368989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:26.369734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.370018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.370344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.371216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:26.371533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:26.373040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.393626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 464/926] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:26.398896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:26.478314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:26.513311 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc182a6-2a90-4e22-abbc-d9f4cc5afb34 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:26.556968 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:26.557764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.558068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.558321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.569701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:26.571143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:26.592210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:26.593157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:26.594072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:26.594072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:26.594072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:26.594072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:26.594072 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 463/927] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:26.599292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:26.650350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:26.721219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65e0b6-588a-4009-ad23-e954182ec959 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:26.761085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:26.761748 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.762028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.762302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.763022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:26.763327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:26.764726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd1b19-91c4-4c00-911e-79a68b7ccd0d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.777702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 465/928] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:26.783579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:26.895367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812a80-d006-4aba-b664-1e6a2d65e132 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:26.903302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237abbe-6cfd-4099-9b6e-4d6692ed9984 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.903593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237abbe-6cfd-4099-9b6e-4d6692ed9984 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.903866 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237abbe-6cfd-4099-9b6e-4d6692ed9984 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.908508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 466/929] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:26.911554 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.911842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.912121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.922116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:26.931779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:26.933009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:26.935032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:26.936204 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:26.936625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:26.936978 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:26.946132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:26.947835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:26.964827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:26.965785 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:26.966904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 464/930] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:26.975269 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.035954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 467/931] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:27.062144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.093126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f2a32-76eb-4270-a6c6-282ffba360eb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.117199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.136280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.137001 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.137827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.138107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.138897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:27.139205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.140653 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.164646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 465/932] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:27.166165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.232687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.240072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe6b1-153a-4ec8-9701-a16efc80bfcd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.281018 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.282135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.282679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.283571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.284586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.285978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-2f5b1217-5eef-4528-8c2a-730feac03c66 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:27.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:27.299457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:27.300303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.305007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 468/933] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:27.306868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8b6-3752-4705-85b5-77ad8b2b1c79 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.307161 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8b6-3752-4705-85b5-77ad8b2b1c79 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.307438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5f8b6-3752-4705-85b5-77ad8b2b1c79 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.309089 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 469/934] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:27.313709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5f8b6-3752-4705-85b5-77ad8b2b1c79 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.319046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be8240-83e9-4d4d-9984-9f644a0f1cee None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.362127 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.363175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.363516 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.363792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/41323e4ae8c145cd950933b4a1733063/roles/73adac171d2d42b28c7c3744379a89e7` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.366195 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=41323e4ae8c145cd950933b4a1733063, role_id=73adac171d2d42b28c7c3744379a89e7)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.368029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a6aec-3e40-435e-9a78-5df0ad524c9b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.383435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 466/935] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:27 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/41323e4ae8c145cd950933b4a1733063/roles/73adac171d2d42b28c7c3744379a89e7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:43:27.389977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c814-cb0b-487b-9377-616e89c7a391 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.390316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c814-cb0b-487b-9377-616e89c7a391 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.390529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9c814-cb0b-487b-9377-616e89c7a391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.422170 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-b7b9c814-cb0b-487b-9377-616e89c7a391 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:27.423975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-b7b9c814-cb0b-487b-9377-616e89c7a391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:27.433353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.443805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5f8b6-3752-4705-85b5-77ad8b2b1c79 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.485378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.486625 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.486903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.487354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.488087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:27.488387 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.489838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fff51a-fed6-4403-b003-8ef1aa06848a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.495884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 470/936] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:27.502712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb64d0-58d1-400f-8c03-5a558a8c78b3 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.502995 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb64d0-58d1-400f-8c03-5a558a8c78b3 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.503270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb64d0-58d1-400f-8c03-5a558a8c78b3 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.504959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 471/937] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:27.505773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb64d0-58d1-400f-8c03-5a558a8c78b3 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.509307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 467/938] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:27.510830 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.511632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.512102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.536664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:27.538154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:27.570113 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.637768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb64d0-58d1-400f-8c03-5a558a8c78b3 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.640514 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 468/939] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:27.641415 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.695345 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.695975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.696198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.696492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.697199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:27.697504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.698876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.708569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 472/940] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:27.709882 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.743989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.761896 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad388d1a-64d1-459b-b06a-6f1f78e6de03 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.806626 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.807264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.807549 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.807820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.808669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.810324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.811416 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: WARNING keystone.server.flask.common [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:43:27.831555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.sql.core [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:43:27.831555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:43:27.831555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [parameters: {'id': '632d7ecdc30e4d7d956d0b9e1d300076', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:43:27.831555 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86704) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 632d7ecdc30e4d7d956d0b9e1d300076.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 632d7ecdc30e4d7d956d0b9e1d300076. May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application conn.query(q) May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:43:27.839121 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result.read() May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 19:43:27.840057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application next(self.gen) May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application yield resource May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application next(self.gen) May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise self.value May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application session.commit() May 19 19:43:27.841028 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.session.flush() May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application compat.raise_( May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:43:27.842951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:43:27.843842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application conn.query(q) May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application result.read() May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:43:27.846236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [parameters: {'id': '632d7ecdc30e4d7d956d0b9e1d300076', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.847340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:27.850035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 632d7ecdc30e4d7d956d0b9e1d300076. May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 469/941] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 19:43:27 2022] POST /identity/v3/projects => generated 262 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:27.852173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.859057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1636e-99a4-427d-957e-109e4454d425 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:27.871289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:27.897953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:27.898775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:27.899051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:27.899326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:27.899971 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:27.901375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:27.915707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:27.916838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:27.918158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 473/942] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:27.919028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:27.976016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ea4821-1278-4e12-9830-64efa841c4ce None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.015372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.016030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.016307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.016651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.017500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] RBAC: Adding query filter params (interface=public, service_id=a73f2f90918249c3a0004539719d58b9, region_id=RegionOne) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:28.017825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.019377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.021211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:28.032445 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 470/943] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/endpoints?service_id=a73f2f90918249c3a0004539719d58b9&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:28.032445 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:28.045136 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779cd2b-aec8-4a55-8d6f-c45be5a61913 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.097184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.098718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.099310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.099771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.112022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.114567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:28.140749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:28.143244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 474/944] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:28.145411 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:28.163589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:28.181972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee314c0-c55f-4ee5-a91f-bf18a4273154 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.220692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.221435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.221818 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.222199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.223018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:28.223430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.225012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.230246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 471/945] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:28.231162 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:28.264091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ae1db-b1fc-4227-aee4-c95f202b27dd None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.303568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.304967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.305580 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.306184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.307646 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.309346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.324438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 475/946] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:28 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:43:28.324438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:28.336725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:28.369802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f6d05-0bbc-4835-b4b1-13e501b6654d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.422213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.423969 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.424438 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.424877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.425770 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:28.426430 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.428649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.458898 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89289ecd-e484-40f4-b4d6-9893ab27c295 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.459489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 472/947] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:28.516509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.517695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.517979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.518322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.519076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:28.519390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.520953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209ad6f-3709-47cf-b64c-cf95ba5385c8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.531903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 476/948] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:28.534854 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:28.548003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:28.670988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e16bda-eb21-42f7-85cb-7fecc5e7d26c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:28.709082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:28.709950 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:28.710064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:28.710286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:28.711049 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:28.712683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba976e-613b-4953-be6b-35e1b0531995 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:28.740475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 473/949] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:29.680373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae7c2-77e5-4cb6-b6bb-6db60c8e7c5e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:29.680944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae7c2-77e5-4cb6-b6bb-6db60c8e7c5e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:29.680944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ae7c2-77e5-4cb6-b6bb-6db60c8e7c5e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:29.682530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 477/950] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:29.688076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:29.688499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:29.688842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:29.707475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:29.709130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:29.778139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:29.793126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 474/951] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:29.820953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ae7c2-77e5-4cb6-b6bb-6db60c8e7c5e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:29.913788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:29.943107 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ae7c2-77e5-4cb6-b6bb-6db60c8e7c5e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:29.982382 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:29.983204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:29.983521 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:29.983747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:29.994764 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:30.003609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:30.004661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:30.005528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 478/952] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:30.010952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.132870 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07aeab-10ba-4008-8182-6020c22f9523 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:30.170246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:30.170912 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.171193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.180559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.181493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:30.181805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:30.183207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:30.193962 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 475/953] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:30.199090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.307839 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b672a-a82f-466d-a497-aa1aabfc4ff7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:30.345649 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:30.346458 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.346726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.347001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.349078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:30.358709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:30.360190 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:30.379770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:30.382090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:30.384271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:30.384271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:30.384271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:30.384271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:30.384271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 479/954] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:30.389540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.450091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:30.504659 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac07f4-e1bc-43e8-b346-027c3f8bfcc0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:30.541631 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:30.542304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.542583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.542854 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.543586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:30.543895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:30.545307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:30.559762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 476/955] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:30.566326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.682404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a8800-328d-43be-9721-2e1b1624477d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:30.721075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:30.722262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.722790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.723260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.731629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:30.733380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-213b6f9f-5963-4f82-b9fd-d68c4d64a9bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:30.749961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:30.750973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:43:30.752086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:30.754933 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 480/956] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:30.761021 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.788567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13fcd-6222-45ea-9255-dc3045b4f402 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.788925 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13fcd-6222-45ea-9255-dc3045b4f402 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.789128 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f13fcd-6222-45ea-9255-dc3045b4f402 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.791195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 481/957] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:30.800003 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.800349 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.800586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.820907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:30.822133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:30.871516 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:30.904428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29af6ac-b41f-4845-9c3d-a6d29169f809 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:30.904638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 482/958] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:30.919253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:30.923090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:30.947158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:30.947837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.948120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.948414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.949577 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:30.949889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:30.951511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9783-a9d2-4450-ac97-dc8612db4c8e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:30.976281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 477/959] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:30.977652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277c10-2193-4bc9-ac39-0690cc89cc62 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:30.977936 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277c10-2193-4bc9-ac39-0690cc89cc62 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:30.978205 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95277c10-2193-4bc9-ac39-0690cc89cc62 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:30.979763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 478/960] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:30.982746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95277c10-2193-4bc9-ac39-0690cc89cc62 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.050391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a794c-64bf-48d9-ac3a-60c59726c973 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.088784 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.089626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.089904 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.090173 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.090826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.092198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-79853aa5-f2f4-46d9-8958-d8b4cd2b16f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:31.104930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:31.106008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.106847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 483/961] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:31.108037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.108037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.108037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.112713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95277c10-2193-4bc9-ac39-0690cc89cc62 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.129114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:31.130306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:31.156832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.158898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.159564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.159853 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.171422 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.205312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.206623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 484/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:31.207464 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.241236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.242871 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13d767-8dc4-4906-b643-948b09516476 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.310639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cca466-85ed-481c-a7c7-b91430631415 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.347772 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.348457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.348742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.349013 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.349709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:31.350014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.351395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.354616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 479/963] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:31.356007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 485/964] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:31.356246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f611b-6d6b-47d8-a715-0719fed82d8a None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.356703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f611b-6d6b-47d8-a715-0719fed82d8a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.356804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.357289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f611b-6d6b-47d8-a715-0719fed82d8a None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.358985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 480/965] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:31.361810 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f611b-6d6b-47d8-a715-0719fed82d8a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.394728 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.435891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.468716 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f611b-6d6b-47d8-a715-0719fed82d8a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.468997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677842c-6153-4794-8d18-86e838a6e845 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.509340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.509935 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.510220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.510496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.511198 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.512670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.513146 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.513819 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.513985 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: WARNING keystone.server.flask.common [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:43:31.514102 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.514377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.515087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:31.515393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.516907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49cb3a-03a1-4283-8724-3bfe06453326 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.522532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 481/966] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:31.524634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.525043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.525370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.526663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.sql.core [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:43:31.526663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:43:31.526663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [parameters: {'user_id': '36d6a772fb9d46c5ae9c61e79d296c45', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:43:31.526663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86703) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application conn.query(q) May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application result.read() May 19 19:43:31.543608 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:43:31.544778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application next(self.gen) May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application yield resource May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application next(self.gen) May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise self.value May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application session.commit() May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:43:31.545818 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self.session.flush() May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application compat.raise_( May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:43:31.546887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:43:31.547795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application conn.query(q) May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application result.read() May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:43:31.549474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [parameters: {'user_id': '36d6a772fb9d46c5ae9c61e79d296c45', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:31.550325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 19:43:31.551191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:43:31.552129 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.553465 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:31.553597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 486/967] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:43:31 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:31.553597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.553734 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:31.622778 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 482/968] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:31.623737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.665605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.672322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a2ce7-cc8c-4ee9-ba52-d369ae190a3d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.701510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.709378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.710184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.710465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.710736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.722242 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.723910 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.743186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3180-9566-4ecf-a158-a06b3ed237c4 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:31.750334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:31.751403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:31.752512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:31.752512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:31.752512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:31.752512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.756391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 487/969] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:31.757311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.778698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.779284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.779567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.779847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.780612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.781977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.804366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 483/970] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:31.805226 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.871725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682aa2-292d-475a-8b7a-e62cd752efbc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.926727 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.927535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.927814 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.928082 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.932367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83250138-c6de-4da7-bd17-21020cabf8a2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:31.936752 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.938247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:31.953371 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:31.954679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:31.955637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:31.956615 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 488/971] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:31.958179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:31.966583 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:31.984809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:31.985297 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:31.985667 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:31.986085 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:31.986932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:31.988436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:32.002775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:32.003931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:32.005373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 484/972] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:32.006166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.093648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46873-4f5d-49ef-ae4f-8fd33d569ce7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.130968 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bb41-3fff-480b-b123-2fba34fd9ba1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.133323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.134074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.134358 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.134629 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.135748 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.136058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.137494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.155498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 489/973] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:32.155498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.175606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.176533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.176696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.176938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.177796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.178107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.179577 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.228442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 485/974] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:32.228442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.232697 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:32.276636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:32.294942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059baa6-ba85-41da-9167-3b5a8fa981df None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.346254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.347281 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.347875 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.348377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.349359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.349917 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.351565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.351698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec152-6c82-41a1-8ff9-29487245a0db None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 490/975] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:32.376758 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.411992 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.412931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.413211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.413483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.421836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.423762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:32.439737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:32.440775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:32.441848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 486/976] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:32.446875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.535239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e76eca-20f8-425f-b2e9-fb81206f1104 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.576896 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:32.581860 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:32.587346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca2948-1f7f-4664-b9a9-a65435176048 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.590547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.590547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.590547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.590547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.590938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.591200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.593706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.600393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 491/977] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:32.640078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.640805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.641089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.641367 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.642570 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.642917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.644435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7edfa-7596-4a60-9b17-d6661b072465 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.662240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 487/978] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:32.667325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:32.805977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2d677-f95c-4865-95b4-224c2205c548 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:32.843790 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:32.844498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:32.845324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:32.845729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:32.846563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] RBAC: Adding query filter params (role.id=73adac171d2d42b28c7c3744379a89e7, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=41323e4ae8c145cd950933b4a1733063) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:32.846871 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:32.848306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d474dbe-ed9b-40c8-990b-7f3aead3c159 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:32.854810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 492/979] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:32 2022] GET /identity/v3/role_assignments?user.id=41323e4ae8c145cd950933b4a1733063&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=73adac171d2d42b28c7c3744379a89e7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:34.241128 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd8e3-5245-433f-af66-e761a9aa92a0 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.241891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd8e3-5245-433f-af66-e761a9aa92a0 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.241891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dd8e3-5245-433f-af66-e761a9aa92a0 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.243264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 488/980] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:34.248811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.249210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.249400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.269046 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:34.270575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:34.326050 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:34.345664 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 493/981] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:34.368231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd8e3-5245-433f-af66-e761a9aa92a0 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:34.484675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dd8e3-5245-433f-af66-e761a9aa92a0 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:34.522142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:34.522872 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.523153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.523429 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.524154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:34.525561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:34.538539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:34.539583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:34.545927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 489/982] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:34.550270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:34.697917 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad4b61-ecb7-4476-a88f-b74c2af012e0 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:34.740691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:34.741613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.741896 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.742172 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.742885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:34.743193 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:34.744600 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2febf6-f185-4fa8-87e3-d0c2f26e80e9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:34.754858 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 494/983] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:34.755871 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:34.783212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:34.868099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd34ba-169c-4a2c-a097-1d9c98881f8c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:34.893473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301d907-ad41-4ff1-b9b8-f7d0ca00022a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.893765 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301d907-ad41-4ff1-b9b8-f7d0ca00022a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.894041 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301d907-ad41-4ff1-b9b8-f7d0ca00022a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.895577 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 495/984] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:34.900553 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.900915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.901104 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.904107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:34.904796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:34.905076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:34.905453 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:34.906242 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:34.906548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:34.909500 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:34.915991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 490/985] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:34.919078 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:34.920738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:34.921481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:34.984874 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:35.019420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 496/986] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:35.035045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:35.043470 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:35.052653 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf83ee-f4cc-48f8-8082-d2e26aaa4abc None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.092152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.092861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.093147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.093792 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.094604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] RBAC: Adding query filter params (interface=admin, service_id=a73f2f90918249c3a0004539719d58b9, region_id=RegionOne) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:35.094921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.096641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.106537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 491/987] 10.222.0.13 () {56 vars in 1249 bytes} [Thu May 19 19:43:34 2022] GET /identity/v3/endpoints?service_id=a73f2f90918249c3a0004539719d58b9&interface=admin®ion_id=RegionOne => generated 193 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:35.107926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:35.171643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846df3a-55f7-47a7-a791-b97cf0631210 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.210943 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.211611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.211889 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.212161 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.212920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.214298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ff6f1-fcb6-4bda-b0e5-c49571c6d55b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.230025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 497/988] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:35 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:43:35.231164 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b399b64-55e3-44a0-92fd-a337b571cfd8 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.231435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a4cc-a89c-4441-b1c7-6ab464a7d7f7 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.231724 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a4cc-a89c-4441-b1c7-6ab464a7d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.231998 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824a4cc-a89c-4441-b1c7-6ab464a7d7f7 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.233267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 498/989] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:35.239477 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.239763 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.240042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.259300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:35.260590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:35.293084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.294397 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.294836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.295317 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.296309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.298338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.301463 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:35.316459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 492/990] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:35 2022] GET /identity/v3/services => generated 1727 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:43:35.351091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 499/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:35.375137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:35.392222 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:35.504327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665cb45-97f6-408d-a2ea-2c9295a11d0c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.551740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.552494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.552775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.553052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] PATH_INFO: `/v3/services/image` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.553717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.555084 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:35.567821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:35.569096 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:35.569929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 493/992] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:43:35 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:35.575139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:35.695310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bfc96-60a8-4789-802a-61f4b1a8fceb None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.731373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.732047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.732338 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.732641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.733355 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] RBAC: Adding query filter params (name=image) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:35.733663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.735134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b2e5f9-3c9a-4a7e-a1af-60dced3696e4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.745785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 500/993] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:35.752065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:35.838720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:35.867724 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdb818-eb9e-4b9a-9827-7540c6cde32c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:35.905067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:35.905744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:35.906016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:35.906287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:35.907026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] RBAC: Adding query filter params (type=image) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:35.907329 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:35.908717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d8c9e5-148c-4d08-ae69-78e6d31bca18 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:35.913977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 494/994] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:37.536966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a1a8-340f-4774-b123-129fde76933c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.537520 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a1a8-340f-4774-b123-129fde76933c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.537520 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521a1a8-340f-4774-b123-129fde76933c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.539070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 501/995] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:37.544066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58c5b-5466-4ac4-8280-29199f85fab0 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.544571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58c5b-5466-4ac4-8280-29199f85fab0 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.544657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58c5b-5466-4ac4-8280-29199f85fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.562109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-79b58c5b-5466-4ac4-8280-29199f85fab0 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:37.563856 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-79b58c5b-5466-4ac4-8280-29199f85fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:37.641158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 495/996] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:37.664051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521a1a8-340f-4774-b123-129fde76933c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:37.684167 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363590-6d04-43c7-b451-e311e9e20363 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.684516 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363590-6d04-43c7-b451-e311e9e20363 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.684803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363590-6d04-43c7-b451-e311e9e20363 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.686200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 496/997] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:37.691499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.691788 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.692125 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.707852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:37.709640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:37.710910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:37.783727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:37.786526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521a1a8-340f-4774-b123-129fde76933c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:37.791890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 497/998] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:37.815459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:37.833768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:37.834584 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.834865 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.835140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.835803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:37.837242 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:37.849540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:37.850527 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:37.851354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 502/999] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:37.857638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:37.934157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e3751-f117-4a1b-8c0a-cf50091ecb13 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:37.977760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:37.978019 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:37.979012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:37.979304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:37.979680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:37.980982 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:37.982447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919508-e732-4d24-832c-c5a486c083c2 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:37.982835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:37.999925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:38.001610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:38.008426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 498/1000] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:43:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:38.009246 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.023347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.024014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.024287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.024591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.025300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] RBAC: Adding query filter params (name=compute) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:38.025605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.026978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.033042 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 503/1001] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:38.038913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.092545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:38.136713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6f6f-4791-47d1-b3c6-129b9db53d5f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.160165 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1725b-0cee-44bd-9025-4b8422cf7c6e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.176756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.177420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.177699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.177977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.178685 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:38.178990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.180347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11b372-fdfc-4346-bb25-6e33afb9c679 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.185345 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 499/1002] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:38.186615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41f2b-dee7-4224-96d6-16c93f90da78 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.186906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41f2b-dee7-4224-96d6-16c93f90da78 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.187175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a41f2b-dee7-4224-96d6-16c93f90da78 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.189301 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 500/1003] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:38.191433 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a41f2b-dee7-4224-96d6-16c93f90da78 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.201044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.201711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.201988 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.202256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.202960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] RBAC: Adding query filter params (type=compute) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:38.203300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.205274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6994b-1e68-4e7b-9f58-a0186b95d50c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.210372 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 504/1004] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:38.211618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.211907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.212435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.228296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:38.229454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:38.251449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:38.299652 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 505/1005] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:38.300658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.307123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a41f2b-dee7-4224-96d6-16c93f90da78 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.337496 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:38.347099 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.347737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.348024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.348302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.349064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:38.349379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.350769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.356299 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 501/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:43:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:38.357159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.412004 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd126a91-c9b2-47df-9550-4d07e1782ba7 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.449177 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.449963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.450116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.450396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.451186 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] RBAC: Adding query filter params (interface=public, service_id=236a321b12aa46bcb2eefceff29136d9, region_id=RegionOne) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:38.451506 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.452978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.459320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 506/1007] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:38 2022] GET /identity/v3/endpoints?service_id=236a321b12aa46bcb2eefceff29136d9&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:38.460150 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.478308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc562a-9c2b-4990-99fd-af28b9fa81b1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.483646 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:38.523750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.524442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.524714 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.524989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.525713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.527074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.541639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 502/1008] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:38 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:38.542306 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:38.577552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fa2b-3a0a-4c8c-a4b9-cf92135e906f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.609551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:38.630990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.631807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.632182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.632528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.633270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.634705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8903d8-6de3-4f7f-ba5e-b8e94ede090d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.661103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 507/1009] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:38.686680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ab22c-10a5-4252-a3c4-616dee77f0a6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:38.738022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:38.738710 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:38.738995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:38.739257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:38.739956 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:38.741663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2d9c9-efb5-4181-a9aa-2b8652151d1b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:38.749353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 503/1010] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:38 2022] GET /identity/v3/services => generated 1957 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:43:40.928059 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677ff2-35c4-4920-8b91-99ce924e1ded None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:40.928701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677ff2-35c4-4920-8b91-99ce924e1ded None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:40.928701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6677ff2-35c4-4920-8b91-99ce924e1ded None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:40.930087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 508/1011] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:40.935542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a3f0-b226-4a5a-9580-8330a2da1491 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:40.936008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a3f0-b226-4a5a-9580-8330a2da1491 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:40.936093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3663a3f0-b226-4a5a-9580-8330a2da1491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:40.958064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-3663a3f0-b226-4a5a-9580-8330a2da1491 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:40.958450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da6f4-86f0-46d9-bb70-d978113f7b46 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:40.958762 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da6f4-86f0-46d9-bb70-d978113f7b46 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:40.959041 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da6f4-86f0-46d9-bb70-d978113f7b46 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:40.959390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3663a3f0-b226-4a5a-9580-8330a2da1491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:40.960591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 509/1012] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:40.965914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:40.966204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:40.966481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:40.992754 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:40.994167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:41.042697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 504/1013] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:40 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:41.044148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e059-5a1d-48b5-bc1e-196d505e3cc9 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.044456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e059-5a1d-48b5-bc1e-196d505e3cc9 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.044736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e059-5a1d-48b5-bc1e-196d505e3cc9 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.046018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 505/1014] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:41.051497 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.051780 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.052054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.075672 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:41.076898 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:41.078179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 510/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:40 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:41.078179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.091047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.115420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.161001 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 506/1016] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:41 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:41.161891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.204151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26829832-c94d-41d8-80a5-659e47facfa8 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.240081 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.241153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.241435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.241711 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] PATH_INFO: `/v3/services/image` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.242366 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.243782 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:41.255936 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:41.257162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.258063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 511/1017] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:41.258921 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.272651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867589e-ebda-4496-803b-1edaba59f80c None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.308015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.308832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.309111 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.309381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.310544 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.312124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.313929 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 507/1018] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:41.314694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.346232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.357256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.382025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06745037-0fc6-4693-a6ed-e4bd40eeb7d6 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.417471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.418283 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.418562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.418838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.419497 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.420951 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.423229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53a216-abed-4c70-9e58-2947da7c2052 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:41.433981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:41.435563 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:41.436737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 512/1019] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:41.437867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.463927 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.464538 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.464823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.465094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.465803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] RBAC: Adding query filter params (name=image) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:41.466107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.467501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.474015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 508/1020] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:41.474889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.543951 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5f6c1-f20a-4ada-a60a-6bddd480b679 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.578244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.579454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.579740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.580014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.580773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.582236 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.585186 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe4a544-ad37-4d84-b979-cd4a5683b027 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.585950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.587000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.627683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 513/1021] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:43:41 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:41.627683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.650873 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.651636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.651928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.652197 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.652964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:41.653271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.654706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.666032 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 509/1022] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:41.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.760653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7ad98-cbcd-4da7-9e30-8e84dc7c6db5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.793795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05144a-ce9a-4aa0-85ff-c732581853b1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.801115 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.801789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.802073 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.802351 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.803072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] RBAC: Adding query filter params (type=image) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:41.803424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.804922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.810897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 514/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:41.816822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.840923 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.841332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.841624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.841899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.843414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:41.843728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.845234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:41.850891 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 510/1024] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:41.857176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:41.876856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.900928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:41.934919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f8c6f-f57f-4fb7-ad92-1ee9b5191658 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.982025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:41.982694 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:41.982817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168ad37-e767-442b-92dc-ba99bdeb9c45 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:41.983233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:41.983233 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:41.984049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] RBAC: Adding query filter params (interface=public, service_id=4e2ba87df70841d4b6e49cb9e56e2be8, region_id=RegionOne) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:41.984365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:41.985829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:42.005955 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 515/1025] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:43:41 2022] GET /identity/v3/endpoints?service_id=4e2ba87df70841d4b6e49cb9e56e2be8&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:42.007024 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:42.027055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:42.027630 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:42.027906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:42.028178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:42.028946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:42.030300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b5d78-5623-4540-b8ec-8b32ee80ab76 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:42.060426 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 511/1026] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:42.133354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cb7f4-df9f-42f6-9e53-2e15b0c46637 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:42.152663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:42.185292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:42.185977 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:42.186259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:42.186533 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:42.187231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:42.188734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb635a-f80a-493e-85aa-ce816557f0c9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:42.212501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 516/1027] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:43:42 2022] GET /identity/v3/services => generated 1957 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:43:43.955906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469499-fd57-4776-b7b2-e1818a05e459 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:43.956518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469499-fd57-4776-b7b2-e1818a05e459 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:43.956609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5469499-fd57-4776-b7b2-e1818a05e459 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:43.958010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 512/1028] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:43.966392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:43.966392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:43.966392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:43.988467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:43.989922 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:44.066355 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 517/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:44.096117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469499-fd57-4776-b7b2-e1818a05e459 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.191444 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:44.210651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5469499-fd57-4776-b7b2-e1818a05e459 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:44.249214 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:44.250823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.251134 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.251399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.252843 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:44.259265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0962b6-067c-44fc-8f53-b7dc5f4d2167 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:44.261359 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 513/1030] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:44.267816 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.385911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0bfa6-e6dc-45cb-966e-4e5e9948969b None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:44.389518 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:44.425891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:44.426567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.426845 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.427119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.427850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:44.429343 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d241f94-8270-4733-9e3a-931a938e268d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:44.435817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a66ba-caac-4675-961a-e5e6a194cbce None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.436116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a66ba-caac-4675-961a-e5e6a194cbce None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.436392 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a66ba-caac-4675-961a-e5e6a194cbce None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.447061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 514/1031] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:44.447061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.447061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.447061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.460639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 518/1032] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:43:44 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:43:44.468686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:44.468686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:44.533255 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5002-bf94-44eb-ba45-e0762ba608cd None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.533578 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5002-bf94-44eb-ba45-e0762ba608cd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.533827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5002-bf94-44eb-ba45-e0762ba608cd None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.535375 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 519/1033] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:44.540934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.541224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.541507 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.547439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 515/1034] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:44.561564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:44.563641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:44.572137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.647550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 520/1035] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:44.670394 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.675320 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:44.695293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7ed43-4d84-41f9-86a1-9ebc004139e6 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:44.743398 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:44.744573 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.745070 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.745416 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.757997 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:44.766984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:44.768017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:44.769126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 516/1036] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:44.774166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.787500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:44.800725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5b48-2738-47cc-afe7-787befe12b8e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:44.841712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:44.842759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.843138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.843313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.844105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:44.845785 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:44.858067 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:44.859206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:44.860210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 521/1037] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:44.866989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.889998 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6ddb-9c5e-42ef-aea5-92e6bf8d5ca4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:44.930571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:44.931256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:44.931526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:44.931804 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:44.932587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:44.932900 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:44.934275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:44.943746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 517/1038] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:44.950747 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:44.963265 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:44.993927 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19dbd2-9b22-467a-afe2-97a6cb5113e8 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.030915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.032113 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.032441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.032721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.033393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] RBAC: Adding query filter params (name=image) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:45.033701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.035102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.048262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 522/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:45.050514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.077658 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:45.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507636-616d-4c5d-b55b-3bef8f07c4bc None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.121036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.121985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.122307 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.122590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.136091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.137593 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:45.158633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:45.159706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 518/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:45.166137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.197840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822f618-51d2-4fc5-bce1-c8609771a2ec None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.238302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.239085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.239369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.239645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.240439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] RBAC: Adding query filter params (type=image) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:45.241012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.242419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.244558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:45.253727 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 523/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:45.254862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.321796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c0d6-c42e-402a-9533-8689f2de038e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.366480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.367141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.367418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.367689 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.368438 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:45.368753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.370115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.372374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:45.385749 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 519/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:45.392044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.397320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a86b8-054d-465a-879a-6ee2b424e206 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.439373 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.440182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.440467 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.440742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.441487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.443028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.476434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 524/1043] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:43:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:43:45.528141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50654-cfc2-4087-b021-e944817b9ae5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.553272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:45.589483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.590325 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.590608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.590881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.603073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.604596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:45.626964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:45.628034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:45.629783 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:45.632805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 520/1044] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:43:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:45.635462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.773707 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b90771-0235-47ba-ad81-6ada853608ca None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:45.823305 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:45.823931 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:45.824219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:45.824524 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:45.825356 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:45.825677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:45.827114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:45.837346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:45.852293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 525/1045] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:45.857917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:45.984199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8dc7a-b63f-4c16-ba04-fc9cdf1d8194 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:46.021528 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:46.022175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:46.022511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:46.022791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:46.071429 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:46.102334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:46.111343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:46.248046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 521/1046] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:43:46.253934 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:46.368514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840702be-4c5f-4b93-b681-69af5114e9a3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:46.408058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:46.408887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:46.409174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:46.409455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:46.420528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:46.422573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:46.442842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:46.443806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:46.444737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:46.444737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:46.444737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:46.444737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:46.448809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 526/1047] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:46.449708 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:46.538578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:46.566897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecfdd4-fb3d-499b-a55e-b80a68e5296c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:46.609737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:46.610422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:46.610698 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:46.610960 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:46.611687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:46.611995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:46.613399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:46.626753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 522/1048] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:46.632960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:46.684937 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:46.744002 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8bb1b-3b7c-4fff-b6e6-8cc1d575ac78 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:46.780804 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:46.781807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:46.782093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:46.782478 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:46.795350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:46.797379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:46.811742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:46.813822 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:46.814733 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:46.815701 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 527/1049] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:46.819806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:46.933931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a54b7-b820-47e3-9d76-68f5478e51aa None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:46.982022 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:46.982683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:46.982966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:46.983241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:46.984036 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:46.986784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:46.987352 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e2a9-9c71-4ec0-865d-162222e32863 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:46.989061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:47.003935 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 523/1050] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:47.009888 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.030178 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7712c-f4e0-45f9-b537-728a7a50a423 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.030587 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7712c-f4e0-45f9-b537-728a7a50a423 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.030731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7712c-f4e0-45f9-b537-728a7a50a423 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.033459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 524/1051] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:47.038244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.038532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.038813 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.060029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:47.061293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:47.131844 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c3d2-4849-4c95-9e93-418e7bacf15f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:47.137045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 525/1052] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:47.142656 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:47.160336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.171133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:47.172567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.172567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.172870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.173670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] RBAC: Adding query filter params (role.id=73adac171d2d42b28c7c3744379a89e7, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:47.174044 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:47.175436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:47.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 528/1053] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=73adac171d2d42b28c7c3744379a89e7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:47.277825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba817d-b139-47b4-8b2b-8aed56b256b0 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:47.309198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:47.323765 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:47.324605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.324888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.325158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.338237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:47.346053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:47.347057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:47.347922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:47.352471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 526/1054] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:47.353391 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.462928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0261fc-454a-46ff-a86c-132feeb7e65f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:47.499047 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:47.499695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.500550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.500823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.501565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:47.501874 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:47.503247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:47.517069 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 529/1055] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:47.518078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.629782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2701898-71ae-4667-817a-999b8a09317a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:47.669955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:47.670869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.671154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.671538 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] PATH_INFO: `/v3/users/716f68eeaada47a4814731b6933ef12c` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.688227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] RBAC: Authorizing `identity:get_user(user_id=716f68eeaada47a4814731b6933ef12c)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:47.689703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2d948-71eb-43cc-8a73-9940b7242136 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:47.691605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 527/1056] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/users/716f68eeaada47a4814731b6933ef12c => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:47.697152 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.736924 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:47.810987 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667c9a9-1b3f-4dae-9ebe-9cf16bcad189 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:47.839964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8147-8144-4aca-b261-da4c10e2cd90 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.840249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8147-8144-4aca-b261-da4c10e2cd90 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.840572 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac8147-8144-4aca-b261-da4c10e2cd90 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.842074 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 528/1057] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:47.846919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.847200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.847472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.852838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:47.853490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:47.853771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:47.854043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:47.865013 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:47.866505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:47.908781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:47.928819 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:47.931096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:47.945435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 529/1058] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:47.968360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:47.974053 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:48.067796 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 530/1059] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:43:48.068949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:48.099949 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d1e57-d0bf-4ccd-9b6b-68f5dbf485b3 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:48.138962 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:48.139604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:48.139883 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:48.140154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:48.186447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924eb04-2fce-499b-ad16-383b6ee8a64a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:48.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:48.218257 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:48.220294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:48.228950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:48.230058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:48.230240 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:48.230501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:48.231914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:48.233354 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:48.240975 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 531/1060] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:43:48 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:48.242486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:48.245472 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:48.338613 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 530/1061] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:43:48.344937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:48.368525 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b789f97-721d-48d7-8344-c1aaf90e87bb None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:48.409809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:48.410502 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:48.410787 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:48.411090 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:48.411912 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=12944e2c9ea643529d2cd6b72c7517a4, user.id=716f68eeaada47a4814731b6933ef12c) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:48.412225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:48.413739 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:48.420833 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 532/1062] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:48 2022] GET /identity/v3/role_assignments?user.id=716f68eeaada47a4814731b6933ef12c&scope.project.id=12944e2c9ea643529d2cd6b72c7517a4&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:48.435686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:48.477676 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc3a08-a264-4172-b4ec-6e4990a0f438 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:48.519731 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:48.520577 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:48.520863 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:48.521139 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:48.534249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:48.535728 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2b980710-ec21-4261-a035-9eb6945006c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:48.556281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:48.557303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:48.564308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:48.564308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:48.564308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:48.564308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:48.564308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 531/1063] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:48.564906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:48.650197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:48.696167 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ac105-da59-463e-9720-7a9a33eca53b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:48.735738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:48.736323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:48.736636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:48.736950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:48.737750 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:48.737950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:48.739332 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc44772-3a8c-498f-ab1c-0ddc0c31c08a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:48.752377 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 533/1064] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:49.480092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654c03-1e24-4523-b0fd-4e3bea06364e None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:49.480644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654c03-1e24-4523-b0fd-4e3bea06364e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:49.480756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654c03-1e24-4523-b0fd-4e3bea06364e None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:49.482132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 532/1065] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:49.487597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:49.487999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:49.488309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:49.506723 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:49.507967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:49.577006 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 534/1066] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:49.599622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654c03-1e24-4523-b0fd-4e3bea06364e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:49.710842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654c03-1e24-4523-b0fd-4e3bea06364e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:49.735895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:49.747596 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:49.748424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:49.748730 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:49.749008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:49.760310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:49.769556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:49.770681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:49.771543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 533/1067] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:49.776757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:49.833806 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:49.886162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1b63a-1cd7-4254-9bb4-351eb517c38e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:49.920815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:49.921591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:49.921882 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:49.922214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:49.923194 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:49.923321 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:49.924702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:49.934049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 535/1068] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:49.939634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.050871 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69945ce6-e808-4528-b6bd-87d757d666d9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.086965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.087820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.088108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.088381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.099635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:50.101115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:50.118583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:50.119663 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:50.121679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:50.121679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:50.121679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:50.121679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:50.121679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 534/1069] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:50.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.242030 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa833a5-eda8-41a9-8e30-85332de55460 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.260256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:50.279214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.279900 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.280276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.280592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.281639 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:50.282058 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:50.283551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:50.296283 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 536/1070] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:50.302311 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.348457 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:50.419244 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544b11-e90a-42a8-b23f-221a1144e61a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.461436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.462222 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.462498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.462766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.470456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:50.471866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:50.486395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:50.487906 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:50.489211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 535/1071] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:50.493797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.596245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c2272-1051-46e5-8bc4-65c0ebba5145 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.634729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.635420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.635709 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.635964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.636887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:50.637189 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:50.638704 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821f361-0699-47db-a52c-05ebd3024c4d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:50.664870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 537/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:50.666319 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.765735 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:50.773003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061890ce-a092-48fe-a638-fbd7a905942f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.808717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.809519 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.809801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.810076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.821376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:50.822821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:50.824492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c60c-b249-47c9-a1af-2d1b5eb2939c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.824788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c60c-b249-47c9-a1af-2d1b5eb2939c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.825062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c60c-b249-47c9-a1af-2d1b5eb2939c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.826436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 538/1073] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:50.831770 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.832053 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.832325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:50.840995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:50.841926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:50.842826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:50.842826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:50.842826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:50.842826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:50.842826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 536/1074] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:50.848348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.850096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:50.851330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:50.885063 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:50.927575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 539/1075] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:50.950366 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:50.957086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec732dd-faa2-4cae-84ed-f71ac6de18f2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:50.998355 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:50.999014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:50.999292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:50.999808 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.000716 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:51.001023 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.002373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.015303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 537/1076] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:51.020763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.032177 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:51.059506 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05faf52-fde6-4748-a0b0-af58d052574c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.096721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.097511 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.097791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.098062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.108320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:51.116567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:51.117590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:51.118449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 540/1077] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:51.124130 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.130420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c853-9080-4624-8deb-c04fbf058e10 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.158448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:51.167047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.167933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.168239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.168551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.176510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.177950 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:51.192510 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:51.193406 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:51.194294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 538/1078] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:51.199679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.235960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648926-272a-47e1-ae1b-7cfe0df8604e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.273460 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:51.273675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.273675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.273675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.273675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.273979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:51.274232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.275803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.285405 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 541/1079] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:51.291315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.311213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4c850-9fd0-40e9-939b-957a3b2c6a44 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.347836 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.348515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.348800 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.349077 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.349852 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:51.350159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.351583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.363553 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 539/1080] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:51.369764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.383486 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:51.396958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0facd5-ceef-4938-acc0-f9d4d829f675 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.431963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.432883 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.433162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.433436 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] PATH_INFO: `/v3/users/716f68eeaada47a4814731b6933ef12c` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.434660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] RBAC: Authorizing `identity:get_user(user_id=716f68eeaada47a4814731b6933ef12c)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.436112 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.437876 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 542/1081] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:43:51 2022] GET /identity/v3/users/716f68eeaada47a4814731b6933ef12c => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:51.443566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.472979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbc888-95a1-4cf5-a9cf-97a9ca5cca5a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.501470 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:51.510957 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.511840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.512124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.512418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/73adac171d2d42b28c7c3744379a89e7` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.514657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=6196d18cd1b44984a2b1c0602745314f, role_id=73adac171d2d42b28c7c3744379a89e7)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.516995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.538501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 540/1082] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:51 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/6196d18cd1b44984a2b1c0602745314f/roles/73adac171d2d42b28c7c3744379a89e7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:43:51.551139 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cde75-7605-4aa8-aaad-65b597062628 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.590771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.591659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.591940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.592213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.593681 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.595187 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb7c14-7d0e-4f34-aa19-9fe4d102869f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.597051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 543/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:43:51 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:51.603334 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:51.719605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458639c-b518-4db5-8868-ba6c12d7b866 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:51.759276 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:51.760166 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:51.760477 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:51.760761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4/users/716f68eeaada47a4814731b6933ef12c/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:51.762751 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] RBAC: Authorizing `identity:create_grant(project_id=12944e2c9ea643529d2cd6b72c7517a4, user_id=716f68eeaada47a4814731b6933ef12c, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:51.764182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ae58b-b9a3-49c5-a5aa-916f27c86c60 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:51.778004 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 541/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:43:51 2022] PUT /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4/users/716f68eeaada47a4814731b6933ef12c/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:43:53.782290 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacdf5-1746-4370-a47c-3e02c0c0a9c2 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:53.782827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacdf5-1746-4370-a47c-3e02c0c0a9c2 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:53.782827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eacdf5-1746-4370-a47c-3e02c0c0a9c2 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:53.784211 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 544/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:53.789635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:53.790068 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:53.790189 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:53.808268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:53.809491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:53.863607 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:53.889033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 542/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:53.913049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eacdf5-1746-4370-a47c-3e02c0c0a9c2 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:53.941835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.024076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eacdf5-1746-4370-a47c-3e02c0c0a9c2 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.059810 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.060569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.060847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.061118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.071840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-5de3aa3d-d7ff-47fc-ba0d-a651eec4dccb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:54.080693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:54.081798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:54.082638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 545/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:43:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:54.088107 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.099623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770737e-357a-49bf-8842-38ddff1c0ebc None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.099913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770737e-357a-49bf-8842-38ddff1c0ebc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.100192 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770737e-357a-49bf-8842-38ddff1c0ebc None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.101605 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 546/1088] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:43:54.106585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.106869 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.107152 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.124742 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:54.125944 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:54.194689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 547/1089] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:43:54.199469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70fed0-8be9-43e3-be29-c774a2de9ada None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.219023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.233637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.239987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.239987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.239987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.239987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.240981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:54.240981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.240981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.253124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 543/1090] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:54.258777 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.303243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.326856 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897873e5-0ce0-425f-b3d0-3ce4d3d985bc None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.363545 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38960c9-b64d-4ff1-b586-b98544d5b169 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.371480 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.372266 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.372573 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.372845 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.383941 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:43:54.392119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:54.393191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:43:54.394060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 548/1091] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:54.399565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.401249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.402176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.402340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.402612 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.417567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.419007 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:54.436491 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:54.438272 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:54.439348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:54.439348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:54.439348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:54.439348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:54.439348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 544/1092] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:54.445309 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.486200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.518150 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923af879-05f0-4703-828d-7513c721f9b0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.542148 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.552419 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.552701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4685bc3-fd64-4ff3-8b85-4a5b3f0285ae None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.553267 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.553481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.553667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.554439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:54.554746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.556149 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.565531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 549/1093] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:54.571469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.588042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.588651 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.588932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.589206 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.589933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:54.590234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.591588 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.604496 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 545/1094] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:54.610713 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.681952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83baecc9-9180-4cbb-bbdd-38e1c91face9 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.719500 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.719760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365a961-3a7c-4d12-abd6-3975223ef938 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.720383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.720696 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.720978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] PATH_INFO: `/v3/users/716f68eeaada47a4814731b6933ef12c` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.722196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] RBAC: Authorizing `identity:get_user(user_id=716f68eeaada47a4814731b6933ef12c)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.723630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.725296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 550/1095] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/users/716f68eeaada47a4814731b6933ef12c => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:54.730855 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.734831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.755722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.756559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.756842 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.757200 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.765903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.767413 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:43:54.783366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:54.784539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:54.785629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 546/1096] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:54.790275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:54.804695 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.848860 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65dffc-1d94-4311-983c-20817ae5ce6b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.885144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.885779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.886054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.886324 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.902202 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d536ae4-a61c-4d2c-9c70-bbfa7945e3ba None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:54.941308 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:54.941973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:54.942251 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:54.942524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:54.943361 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:54.943675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.945128 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.955946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:54.957435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:54.960536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:54.964562 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 547/1097] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:54.970387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.042733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.062192 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 551/1098] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:43:55.068259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.074159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca0d0-30a1-4697-9858-8317aa4f5d9a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.109425 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.109985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.110264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.110542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.170585 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24f7af-c255-4cff-b8b3-3c01f98a39ec None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.174194 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.182961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.184421 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.208052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.208939 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.209221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.209492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.210676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.212105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.213826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 552/1099] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:55.220263 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.257181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.293938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 548/1100] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:43:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:43:55.300360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.323836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95783ae8-98e6-44fb-a3c7-f29096dca31f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.363310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.363878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.364162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.364459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.365538 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=12944e2c9ea643529d2cd6b72c7517a4, user.id=716f68eeaada47a4814731b6933ef12c) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:55.365850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.367290 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.373408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 553/1101] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/role_assignments?user.id=716f68eeaada47a4814731b6933ef12c&scope.project.id=12944e2c9ea643529d2cd6b72c7517a4&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:55.405184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2dca8-b695-43f0-b96d-aee550ccb383 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.445381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.446199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.446474 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.446743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.458881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.460456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:43:55.479075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:55.480073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:55.481259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:43:55.481259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:43:55.481259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:43:55.481259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:55.481259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 549/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:43:55.491678 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.529734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.607125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf2bac-ac1a-4913-92e1-0b9d7df11c10 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.645395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.646085 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.646367 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.646643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.647381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:55.647688 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.649114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.661586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 554/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:55.667098 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.692841 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.772947 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd871-e9c8-4b78-984c-051543ed56c5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.808972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.809802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.810043 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.810312 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.818184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:55.819629 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:43:55.834141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:43:55.835090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:43:55.836065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 550/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:43:55.841157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:55.946213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:55.954252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5971e-4047-4fb5-b19e-ed8c71b80e0b None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:55.996179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:55.996945 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:55.997231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:55.997501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:55.998285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:55.998599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:56.000094 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afedfa-ed26-40b7-9843-eac521ee42a1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:56.013593 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 555/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:56.019644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:56.109322 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:56.130400 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd02976-9be7-4f0f-93b6-f809b0d15060 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:56.169604 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:56.170274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:56.170557 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:56.170831 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:56.171922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] RBAC: Adding query filter params (role.id=73adac171d2d42b28c7c3744379a89e7, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=6196d18cd1b44984a2b1c0602745314f) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:43:56.172236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:56.173706 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f472-830f-4e44-aa4f-08f2de1cc536 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:56.180243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 551/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:43:56 2022] GET /identity/v3/role_assignments?user.id=6196d18cd1b44984a2b1c0602745314f&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=73adac171d2d42b28c7c3744379a89e7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:43:57.676982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795b68-9adc-4b7f-b963-87007103dfe2 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:57.677543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795b68-9adc-4b7f-b963-87007103dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:57.677543 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8795b68-9adc-4b7f-b963-87007103dfe2 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:57.678984 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 556/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:43:57.684781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:57.685258 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:57.685346 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:57.702476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:43:57.703692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:43:57.771954 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 552/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:43:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:43:57.797337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8795b68-9adc-4b7f-b963-87007103dfe2 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:57.904863 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8795b68-9adc-4b7f-b963-87007103dfe2 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:57.938675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:57.939400 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:57.939674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:57.939942 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:57.941208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:57.942670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a9975-3b8d-48ce-b266-e7a6eb1fda91 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:57.944293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 557/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:43:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:43:57.950029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:43:58.042262 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:43:58.066496 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dfec8-16df-4667-b2e4-b2e0a9360c38 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:43:58.111753 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:43:58.112440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:43:58.112727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:43:58.113002 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:43:58.113829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:43:58.115512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14870b8-365b-4470-915e-1cebd5fdefad None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:43:58.136135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 553/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:43:57 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:00.454561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0817eb-71cc-40d3-aa38-4ec3c0c4e4df None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:00.455144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0817eb-71cc-40d3-aa38-4ec3c0c4e4df None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:00.455144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0817eb-71cc-40d3-aa38-4ec3c0c4e4df None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:00.456498 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 558/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:00.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:00.462169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:00.462343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:00.479310 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:00.480518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:00.549090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 554/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:00.572592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0817eb-71cc-40d3-aa38-4ec3c0c4e4df None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:00.600566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:00.678645 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0817eb-71cc-40d3-aa38-4ec3c0c4e4df None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:00.713264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:00.714028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:00.714299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:00.714566 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] PATH_INFO: `/v3/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:00.716012 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] RBAC: Authorizing `identity:get_role(role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:00.717410 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:00.719404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 559/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:00 2022] GET /identity/v3/roles/afe01609a07c422aa82e18ea2bb45744 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:00.725771 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:00.819619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:00.860075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0700ae-e8cc-4839-906b-9319762c6c4d None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:00.894016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:00.894869 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:00.895150 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:00.895418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] PATH_INFO: `/v3/users/b04d0ca13d3e4bdc85d050032e1c636f` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:00.910758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] RBAC: Authorizing `identity:get_user(user_id=b04d0ca13d3e4bdc85d050032e1c636f)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:00.912204 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:00.913964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 555/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:00 2022] GET /identity/v3/users/b04d0ca13d3e4bdc85d050032e1c636f => generated 286 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:00.919702 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:01.017960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:01.026963 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4745a4-690f-4f22-b13a-a59cf93766fb None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:01.065300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:01.068346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:01.068642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:01.068908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:01.145749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:01.147217 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:01.238221 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:01.248320 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 560/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:01.254867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:01.373409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfe335-03e8-4d87-b23a-e5c4415cf3c9 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:01.395592 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:01.409656 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:01.410501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:01.410784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:01.411048 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:01.412199 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:01.413658 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05cf79-3951-48dd-be06-493051c85878 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:01.415337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 556/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:01 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:01.421954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:01.522220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa0da5-429a-4af8-ab28-86b209ca272c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:01.557268 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:01.557915 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:01.558199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:01.558475 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:01.559305 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=12944e2c9ea643529d2cd6b72c7517a4, user.id=b04d0ca13d3e4bdc85d050032e1c636f) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:01.559616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:01.561117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa71f6-114f-441c-9909-bb5cb9f0d51b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:01.567424 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 561/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:01 2022] GET /identity/v3/role_assignments?user.id=b04d0ca13d3e4bdc85d050032e1c636f&scope.project.id=12944e2c9ea643529d2cd6b72c7517a4&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:44:03.868708 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5044e-5317-4c67-aaf5-4ea015f35644 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:03.869238 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5044e-5317-4c67-aaf5-4ea015f35644 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:03.869238 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba5044e-5317-4c67-aaf5-4ea015f35644 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:03.870722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 557/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:03.877165 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:03.877773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:03.877773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:03.894349 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:03.895544 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:03.947683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:03.964925 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 562/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:03.995932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5044e-5317-4c67-aaf5-4ea015f35644 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:04.124989 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba5044e-5317-4c67-aaf5-4ea015f35644 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:04.160590 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:04.161442 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:04.161725 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:04.162000 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] PATH_INFO: `/v3/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:04.163482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] RBAC: Authorizing `identity:get_role(role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:04.164890 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:04.166266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 558/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:03 2022] GET /identity/v3/roles/afe01609a07c422aa82e18ea2bb45744 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:04.172825 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:04.272049 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407528ad-a701-40be-922e-f0389b7e7293 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:04.307614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:04.308510 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:04.308788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:04.309054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] PATH_INFO: `/v3/users/b04d0ca13d3e4bdc85d050032e1c636f` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:04.310238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] RBAC: Authorizing `identity:get_user(user_id=b04d0ca13d3e4bdc85d050032e1c636f)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:04.311759 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:04.313406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 563/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:04 2022] GET /identity/v3/users/b04d0ca13d3e4bdc85d050032e1c636f => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:04.319181 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:04.377135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:04.422912 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496de2e-dca7-466d-a061-e89470a801f1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:04.458343 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:04.459203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:04.459486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:04.459761 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:04.460985 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:04.462412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da72e-148a-4bee-b601-e03020a321f4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:04.464115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 559/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:04 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:04.471142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:04.532074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:04.578262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491a646-5955-4508-8b9a-2081074d2255 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:04.616836 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:04.617726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:04.618009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:04.618285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4/users/b04d0ca13d3e4bdc85d050032e1c636f/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:04.620550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] RBAC: Authorizing `identity:create_grant(project_id=12944e2c9ea643529d2cd6b72c7517a4, user_id=b04d0ca13d3e4bdc85d050032e1c636f, role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:04.622009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db699c6-5693-4de8-8f01-bd42b49ec4ca None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:04.635158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 564/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:44:04 2022] PUT /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4/users/b04d0ca13d3e4bdc85d050032e1c636f/roles/afe01609a07c422aa82e18ea2bb45744 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:44:06.892479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad328ef-3c7b-49b2-aaef-f7c6aef11c9a None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:06.893027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad328ef-3c7b-49b2-aaef-f7c6aef11c9a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:06.893027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad328ef-3c7b-49b2-aaef-f7c6aef11c9a None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:06.894412 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 560/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:06.900235 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:06.900761 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:06.900850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:06.917911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:06.919117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:06.988446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 565/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:07.017135 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad328ef-3c7b-49b2-aaef-f7c6aef11c9a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:07.125215 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad328ef-3c7b-49b2-aaef-f7c6aef11c9a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:07.148839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:07.161876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:07.162734 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:07.163184 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:07.163321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] PATH_INFO: `/v3/roles/afe01609a07c422aa82e18ea2bb45744` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:07.164801 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] RBAC: Authorizing `identity:get_role(role_id=afe01609a07c422aa82e18ea2bb45744)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:07.166171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:07.167559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 561/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:07 2022] GET /identity/v3/roles/afe01609a07c422aa82e18ea2bb45744 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:07.173485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:07.220022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:07.277504 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7016b-2604-4448-b9b2-bdd3dd317db1 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:07.317685 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:07.318558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:07.318838 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:07.319110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] PATH_INFO: `/v3/users/b04d0ca13d3e4bdc85d050032e1c636f` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:07.320298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] RBAC: Authorizing `identity:get_user(user_id=b04d0ca13d3e4bdc85d050032e1c636f)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:07.321766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:07.323404 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 566/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:07 2022] GET /identity/v3/users/b04d0ca13d3e4bdc85d050032e1c636f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:07.329942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:07.435357 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0a746-9763-4f47-a25f-8eb177b9557b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:07.470637 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:07.471304 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:07.471586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:07.471858 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:07.535097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:07.544469 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:07.546169 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:07.658157 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 562/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:07.664054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:07.769766 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5425615-ad81-49d8-a305-9e88aeceb7ce None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:07.784460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:07.806548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:07.807409 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:07.807691 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:07.807966 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] PATH_INFO: `/v3/projects/12944e2c9ea643529d2cd6b72c7517a4` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:07.809162 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] RBAC: Authorizing `identity:get_project(project_id=12944e2c9ea643529d2cd6b72c7517a4)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:07.810623 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82c169d-fcd7-4183-846c-5a47327b3972 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:07.813039 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 567/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:07 2022] GET /identity/v3/projects/12944e2c9ea643529d2cd6b72c7517a4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:07.819116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:07.913236 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:07.922806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962b1ca-c0da-4086-bf02-dcd24c95681b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:07.957078 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:07.957723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:07.958001 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:07.958273 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:07.959086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] RBAC: Adding query filter params (role.id=afe01609a07c422aa82e18ea2bb45744, scope.project.id=12944e2c9ea643529d2cd6b72c7517a4, user.id=b04d0ca13d3e4bdc85d050032e1c636f) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:07.959393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:07.960821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9476c6a-83d8-44ec-b45d-72628a6f617d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:07.968576 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 563/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:07 2022] GET /identity/v3/role_assignments?user.id=b04d0ca13d3e4bdc85d050032e1c636f&scope.project.id=12944e2c9ea643529d2cd6b72c7517a4&role.id=afe01609a07c422aa82e18ea2bb45744 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:10.220831 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1322-e4d5-4101-904c-934c7d7e8c8c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:10.221450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1322-e4d5-4101-904c-934c7d7e8c8c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:10.221450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1322-e4d5-4101-904c-934c7d7e8c8c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:10.222660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 568/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:10.228172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:10.228919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:10.228919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:10.246063 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:10.247243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:10.315066 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 564/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:10.338895 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1322-e4d5-4101-904c-934c7d7e8c8c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:10.444717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1322-e4d5-4101-904c-934c7d7e8c8c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:10.480193 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:10.480930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:10.481213 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:10.481488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:10.482781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:10.484243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a284ef64-b998-4df4-ae66-3bf5164a0bfe None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:10.485940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 569/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:44:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:10.492240 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:10.577079 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:10.593054 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4491-4d61-49b8-a75d-bdf7570737ab None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:10.629930 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:10.630594 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:10.630876 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:10.631150 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:10.631874 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:10.633341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c5451-6c32-47e7-a676-a68ca8f4af75 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:10.668784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 565/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:44:10 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:12.930244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c067-95fe-44cf-aeab-725239af116b None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:12.930803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c067-95fe-44cf-aeab-725239af116b None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:12.930803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c067-95fe-44cf-aeab-725239af116b None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:12.932138 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 570/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:12.937505 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:12.938029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:12.938132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:12.955569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:12.956795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:13.024866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 566/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:13.050489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c067-95fe-44cf-aeab-725239af116b None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:13.102081 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:13.156717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c067-95fe-44cf-aeab-725239af116b None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:13.190306 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:13.191015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:13.191294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:13.191565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:13.192752 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:13.194195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07034eb-5f37-48fb-aaeb-23f4543e695d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:13.195795 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 571/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:44:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:13.202533 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:13.270348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:13.304300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215be95b-6f4b-4f5c-b960-cb4aa304842a None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:13.340117 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:13.340931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:13.341210 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:13.341482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:13.342274 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:13.343686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42fdbe-2fb0-48d5-aa8a-0b8e235efb0c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:13.365646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 567/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:44:13 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:15.716958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83005e-2945-404d-8fb4-0f728d40671a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:15.717531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83005e-2945-404d-8fb4-0f728d40671a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:15.717531 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83005e-2945-404d-8fb4-0f728d40671a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:15.718868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 572/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:15.724241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:15.724680 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:15.724833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:15.742335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:15.743531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:15.816171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 568/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:15.840108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83005e-2945-404d-8fb4-0f728d40671a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:15.944100 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83005e-2945-404d-8fb4-0f728d40671a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:15.970665 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:15.989748 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:15.990546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:15.990829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:15.991210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:16.002309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:16.010202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:16.011369 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:16.012271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 573/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:16.018124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:16.073387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:16.122639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad111a87-8dd3-4f00-8f42-c4320bdbb1e4 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:16.161903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:16.162563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:16.162840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:16.163110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:16.163848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:16.164152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:16.165569 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:16.187530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 569/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:16.196151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:16.302080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fbaee-187d-48b9-bb4d-fcc61ee77c1d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:16.345907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:16.346781 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:16.347060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:16.347335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] PATH_INFO: `/v3/users/aea795cb424e4bd1904faf3652333b18` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:16.361256 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] RBAC: Authorizing `identity:get_user(user_id=aea795cb424e4bd1904faf3652333b18)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:16.362696 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:16.364612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 574/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:16 2022] GET /identity/v3/users/aea795cb424e4bd1904faf3652333b18 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:16.370302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:16.472193 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6401b9-389b-43a4-ac71-12a13d0d3a22 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:16.507413 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:16.530093 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:16.530755 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:16.531040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:16.531316 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:16.604705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:16.606075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:16.708047 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 570/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:16.713907 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:16.749245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:16.818760 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e445925-7ad5-4ea5-b83a-4d17ec0791d4 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:16.852638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:16.853492 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:16.853767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:16.854036 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] PATH_INFO: `/v3/projects/9b49135373ca491090368484c29f13b4` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:16.855266 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] RBAC: Authorizing `identity:get_project(project_id=9b49135373ca491090368484c29f13b4)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:16.856673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793f8e2-06d4-4989-af00-f4a82de30f3a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:16.858894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 575/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:16 2022] GET /identity/v3/projects/9b49135373ca491090368484c29f13b4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:16.865247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:16.882988 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:16.968623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64f326-00f8-4b9f-888c-d74e3073ff40 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:17.002920 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:17.003579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:17.003861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:17.004136 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:17.005027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=9b49135373ca491090368484c29f13b4, user.id=aea795cb424e4bd1904faf3652333b18) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:17.005348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:17.006812 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a6c9d-4871-4970-af20-e0aa8a3c267c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:17.013159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 571/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:16 2022] GET /identity/v3/role_assignments?user.id=aea795cb424e4bd1904faf3652333b18&scope.project.id=9b49135373ca491090368484c29f13b4&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:19.298396 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd57e9-164e-480c-85c1-143cb5a8851d None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:19.298923 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd57e9-164e-480c-85c1-143cb5a8851d None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:19.298923 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd57e9-164e-480c-85c1-143cb5a8851d None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:19.300295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 576/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:19.306963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:19.307513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:19.307513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:19.324494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:19.326410 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:19.394781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 572/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:19.423350 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd57e9-164e-480c-85c1-143cb5a8851d None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:19.537967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd57e9-164e-480c-85c1-143cb5a8851d None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:19.539512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:19.574895 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:19.575663 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:19.576462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:19.576735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:19.586882 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:19.594591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:19.595964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:19.597328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 577/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:19.601983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:19.615249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:19.705138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0850a-bd95-4c75-946e-56cc371ba353 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:19.743746 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:19.744423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:19.744723 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:19.744995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:19.745737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:19.746044 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:19.747486 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:19.756305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 573/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:19.762292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:19.875390 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca75bb-02cb-49ff-84c7-49a2a8543ebc None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:19.909315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:19.910183 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:19.910465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:19.910740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] PATH_INFO: `/v3/users/aea795cb424e4bd1904faf3652333b18` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:19.911932 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] RBAC: Authorizing `identity:get_user(user_id=aea795cb424e4bd1904faf3652333b18)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:19.913413 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:19.915035 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 578/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:19 2022] GET /identity/v3/users/aea795cb424e4bd1904faf3652333b18 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:19.921432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:19.993816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:20.024018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d562a9-2add-4499-b529-a67f538b4c92 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:20.059429 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:20.060324 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:20.060697 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:20.060974 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] PATH_INFO: `/v3/projects/9b49135373ca491090368484c29f13b4` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:20.062129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] RBAC: Authorizing `identity:get_project(project_id=9b49135373ca491090368484c29f13b4)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:20.063543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cabda9-af31-4079-ac0f-2ca8260a190e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:20.065289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 574/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:19 2022] GET /identity/v3/projects/9b49135373ca491090368484c29f13b4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:20.071141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:20.103609 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:20.177546 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c3a92-fff2-48ee-b7d2-7c9577357467 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:20.224263 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:20.225168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:20.225450 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:20.225721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] PATH_INFO: `/v3/projects/9b49135373ca491090368484c29f13b4/users/aea795cb424e4bd1904faf3652333b18/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:20.227687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] RBAC: Authorizing `identity:create_grant(project_id=9b49135373ca491090368484c29f13b4, user_id=aea795cb424e4bd1904faf3652333b18, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:20.229160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e4ae3-4162-405e-be0d-15eb1ccf51ee None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:20.244979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 579/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:44:20 2022] PUT /identity/v3/projects/9b49135373ca491090368484c29f13b4/users/aea795cb424e4bd1904faf3652333b18/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:44:22.487118 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837caeb5-7457-435a-9df1-8453a31713e6 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:22.487678 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837caeb5-7457-435a-9df1-8453a31713e6 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:22.487678 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837caeb5-7457-435a-9df1-8453a31713e6 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:22.489336 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 575/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:22.494673 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac582430-8618-4105-b324-da44c382e790 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:22.495105 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac582430-8618-4105-b324-da44c382e790 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:22.495231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac582430-8618-4105-b324-da44c382e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:22.512322 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ac582430-8618-4105-b324-da44c382e790 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:22.514307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ac582430-8618-4105-b324-da44c382e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:22.587793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 580/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:22.613071 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837caeb5-7457-435a-9df1-8453a31713e6 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:22.737579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837caeb5-7457-435a-9df1-8453a31713e6 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:22.772895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:22.773684 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:22.773968 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:22.774243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:22.781668 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:22.785501 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:22.793462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:22.795097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:22.796005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:44:22.800362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 576/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:22.801287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac582430-8618-4105-b324-da44c382e790 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:22.822697 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:22.903007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac582430-8618-4105-b324-da44c382e790 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:22.937459 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:22.938107 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:22.938385 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:22.938657 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:22.939389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:22.939693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:22.941244 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:22.950682 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 581/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:22.956673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:23.061600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2256f9c-8b43-48d6-9ea3-f257d1299ad2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:23.095321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:23.096180 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:23.096488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:23.096765 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] PATH_INFO: `/v3/users/aea795cb424e4bd1904faf3652333b18` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:23.098087 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] RBAC: Authorizing `identity:get_user(user_id=aea795cb424e4bd1904faf3652333b18)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:23.099526 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:23.101247 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 577/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:22 2022] GET /identity/v3/users/aea795cb424e4bd1904faf3652333b18 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:23.107910 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:23.195928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:23.215151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e67c5c-175c-42fb-8516-7f9a097c710d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:23.250254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:23.250897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:23.251179 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:23.251456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:23.327357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:23.328793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:23.410614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:23.430134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 582/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:23.436011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:23.507355 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:23.543523 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61f44-48b5-4dcc-99a1-e44c8c8cc582 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:23.581745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:23.582523 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:23.582802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:23.583075 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] PATH_INFO: `/v3/projects/9b49135373ca491090368484c29f13b4` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:23.584268 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] RBAC: Authorizing `identity:get_project(project_id=9b49135373ca491090368484c29f13b4)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:23.585772 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ed10e-4408-49d6-9a99-120146e0a470 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:23.587487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 578/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:23 2022] GET /identity/v3/projects/9b49135373ca491090368484c29f13b4 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:23.594333 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:23.704170 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80bf71-05ea-41b1-9ceb-20a2ed299856 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:23.739316 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:23.739964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:23.740243 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:23.740555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:23.741397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=9b49135373ca491090368484c29f13b4, user.id=aea795cb424e4bd1904faf3652333b18) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:23.741713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:23.743164 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eb9af-26be-4101-b394-f731aac38623 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:23.749532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 583/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:23 2022] GET /identity/v3/role_assignments?user.id=aea795cb424e4bd1904faf3652333b18&scope.project.id=9b49135373ca491090368484c29f13b4&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:26.099089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fc7d5-3409-4987-bec8-35eaba10dec5 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:26.099643 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fc7d5-3409-4987-bec8-35eaba10dec5 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:26.099643 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1fc7d5-3409-4987-bec8-35eaba10dec5 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:26.101064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 579/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:26.106860 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:26.107303 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:26.107403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:26.125714 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:26.126938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:26.172230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:26.207767 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 584/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:26.230067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fc7d5-3409-4987-bec8-35eaba10dec5 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:26.339499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1fc7d5-3409-4987-bec8-35eaba10dec5 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:26.374338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:26.374970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:26.375253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:26.375527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:26.379089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:26.445760 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:26.447215 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:26.546817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 580/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:26.552952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:26.654353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ddac6-4120-4801-a987-7c895b5b89ef None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:26.689266 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:26.690096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:26.690383 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:26.690660 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:26.706590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:26.708079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-bfd2c6bd-a315-4dd3-afaa-063888dcb989 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:26.722989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:26.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:26.725110 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 585/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:44:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:26.730284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:26.767945 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:26.841815 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4059b2-79c0-485c-acf8-f9dd154b1a61 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:26.877011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:26.877661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:26.877939 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:26.878209 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:26.878917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:26.879219 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:26.880605 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5385ca-8975-4dab-9350-36a4a0a5d40a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:26.888208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 581/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:44:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:29.090448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9f44-333a-4ab4-bc6a-a78c99a15540 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:29.090986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9f44-333a-4ab4-bc6a-a78c99a15540 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:29.090986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c9f44-333a-4ab4-bc6a-a78c99a15540 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:29.100700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 586/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:29.107018 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aaf78-a344-4b7e-b575-7716c125d68c None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:29.107524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aaf78-a344-4b7e-b575-7716c125d68c None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:29.107633 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436aaf78-a344-4b7e-b575-7716c125d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:29.124931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-436aaf78-a344-4b7e-b575-7716c125d68c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:29.126137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-436aaf78-a344-4b7e-b575-7716c125d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:29.203921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 582/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:29.226572 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9f44-333a-4ab4-bc6a-a78c99a15540 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:29.276698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:29.346068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c9f44-333a-4ab4-bc6a-a78c99a15540 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:29.387225 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:29.387913 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:29.388197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:29.388501 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:29.389238 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:29.390619 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b462c17-ae72-453d-a5ab-cc3feb1a8706 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:29.413851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 587/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu May 19 19:44:29 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:31.716615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e236a-b971-4521-934c-1bdd316ce0a3 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:31.717153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e236a-b971-4521-934c-1bdd316ce0a3 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:31.717153 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e236a-b971-4521-934c-1bdd316ce0a3 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:31.718544 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 583/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:31.723772 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:31.724209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:31.724323 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:31.741259 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:31.742446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:31.810786 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 588/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:31.834212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e236a-b971-4521-934c-1bdd316ce0a3 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:31.866479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:31.952275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e236a-b971-4521-934c-1bdd316ce0a3 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:31.988970 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:31.989829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:31.990105 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:31.990373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] PATH_INFO: `/v3/domains/e5c562d6abf9437abae52cb6417b121c` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:32.001056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e5c562d6abf9437abae52cb6417b121c)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:32.002472 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fa34d-3b2c-4121-9c07-7885095368a3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:32.004155 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 584/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 19:44:31 2022] GET /identity/v3/domains/e5c562d6abf9437abae52cb6417b121c => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:32.010219 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:32.037736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:32.125456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9a3c9-19df-4cc2-bc1b-304dbd065bfd None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:32.160080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:32.160824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:32.161112 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:32.161387 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:32.162114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:32.163597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7253a87-2f99-4609-9cf7-bc75eef63315 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:32.190973 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 589/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:44:32 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:34.517137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a950d2-b263-43f2-8273-8e576b03b95e None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:34.517701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a950d2-b263-43f2-8273-8e576b03b95e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:34.517701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a950d2-b263-43f2-8273-8e576b03b95e None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:34.519056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 585/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:34.524553 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:34.524962 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:34.525118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:34.542126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:34.543959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:34.622206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 590/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:34.649787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a950d2-b263-43f2-8273-8e576b03b95e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:34.746351 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:34.757295 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a950d2-b263-43f2-8273-8e576b03b95e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:34.792494 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:34.793314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:34.793589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:34.793856 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] PATH_INFO: `/v3/domains/e5c562d6abf9437abae52cb6417b121c` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:34.795011 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=e5c562d6abf9437abae52cb6417b121c)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:34.796488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa241cf-ed95-4659-8661-50ef0a1bf70c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:34.798088 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 586/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 19:44:34 2022] GET /identity/v3/domains/e5c562d6abf9437abae52cb6417b121c => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:34.803911 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:34.814485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:34.907917 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807caa49-a0f4-420b-b655-9f305cde2264 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:34.946590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:34.947257 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:34.947535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:34.947811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:34.948607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:34.950827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f7098-5fef-44c2-a136-204309968501 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:34.978712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 591/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:44:34 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:44:37.313427 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c054-164c-43b9-96c1-daaca459d520 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:37.313932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c054-164c-43b9-96c1-daaca459d520 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:37.313932 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4c054-164c-43b9-96c1-daaca459d520 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:37.315267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 587/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:37.320570 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:37.320970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:37.321123 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:37.339505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:37.340862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:37.400568 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:37.410340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 592/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:37.434835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4c054-164c-43b9-96c1-daaca459d520 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:37.541617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e4c054-164c-43b9-96c1-daaca459d520 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:37.577578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:37.578286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:37.578567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:37.578839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:37.590556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:37.598395 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:37.599463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:44:37.600284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 588/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:44:37.606296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:37.707569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4e69c-3f66-4b8f-b300-46957978c180 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:37.741327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:37.741989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:37.742265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:37.742535 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:37.743290 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:37.743618 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:37.745080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:37.754829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 593/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:37.760718 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:37.779502 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:37.862979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93daf7-dd13-4b27-b167-d9be03e25cfb None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:37.896347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:37.897263 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:37.897544 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:37.897815 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] PATH_INFO: `/v3/users/11087bbc573b4699a3913d975359db41` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:37.912657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=11087bbc573b4699a3913d975359db41)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:37.914097 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:37.915822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 589/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:37 2022] GET /identity/v3/users/11087bbc573b4699a3913d975359db41 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:37.921644 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:37.987561 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:38.033829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609d3ca-1318-41f4-9522-dba44df41f78 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:38.071642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:38.072296 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:38.072606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:38.072884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:38.143168 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:38.144680 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:38.204055 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:38.249765 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 594/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:38.255614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:38.338741 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:38.361754 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f3104-2bf0-4959-9124-9b527e488bd4 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:38.400249 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:38.401057 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:38.401338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:38.401609 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] PATH_INFO: `/v3/projects/faa43f27456748e49b23b5a38a24911b` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:38.402925 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] RBAC: Authorizing `identity:get_project(project_id=faa43f27456748e49b23b5a38a24911b)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:38.404364 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efdc22-0f95-42f7-8662-ba7ebc3873c7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:38.406115 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 590/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:38 2022] GET /identity/v3/projects/faa43f27456748e49b23b5a38a24911b => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:38.415289 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:38.518209 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bdf4e-42fa-4975-bdeb-b8ebade2da93 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:38.553381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:38.554037 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:38.554310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:38.554582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:38.555408 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=faa43f27456748e49b23b5a38a24911b, user.id=11087bbc573b4699a3913d975359db41) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:38.555722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:38.557204 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7ae6-3757-421f-893e-d1911b0ac279 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:38.563381 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 595/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:38 2022] GET /identity/v3/role_assignments?user.id=11087bbc573b4699a3913d975359db41&scope.project.id=faa43f27456748e49b23b5a38a24911b&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:40.786971 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8427-4ebb-44b0-a674-74a6aef7b806 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:40.787616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8427-4ebb-44b0-a674-74a6aef7b806 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:40.787616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b8427-4ebb-44b0-a674-74a6aef7b806 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:40.788937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 591/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:40.794015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:40.794453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:40.794580 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:40.811454 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:40.812680 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:40.828778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:40.882877 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 596/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:40.906286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b8427-4ebb-44b0-a674-74a6aef7b806 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:41.017955 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b8427-4ebb-44b0-a674-74a6aef7b806 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:41.052183 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:41.052908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:41.053191 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:41.053462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:41.064004 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:41.071673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:41.072821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:44:41.073740 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 592/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:41.081978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:41.190841 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9097fcb-1f10-41b1-9b71-ebb28005562c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:41.225298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:41.225961 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:41.226245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:41.226567 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:41.227262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:41.227571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:41.228981 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:41.237335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 597/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:41.243817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:41.248561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:41.348132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fedc5-a88a-4c6a-b802-b5caf3888954 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:41.382881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:41.383744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:41.384025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:41.384297 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] PATH_INFO: `/v3/users/11087bbc573b4699a3913d975359db41` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:41.385508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] RBAC: Authorizing `identity:get_user(user_id=11087bbc573b4699a3913d975359db41)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:41.387017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:41.388671 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 593/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:41 2022] GET /identity/v3/users/11087bbc573b4699a3913d975359db41 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:41.401826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:41.418016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:41.506299 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f51276-6cad-4113-9252-44df4cf84166 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:41.538903 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:41.539753 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:41.540026 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:41.540293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] PATH_INFO: `/v3/projects/faa43f27456748e49b23b5a38a24911b` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:41.541493 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] RBAC: Authorizing `identity:get_project(project_id=faa43f27456748e49b23b5a38a24911b)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:41.543485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26656e-dbe3-4ceb-85bb-773b46f00e9e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:41.545184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 598/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:41 2022] GET /identity/v3/projects/faa43f27456748e49b23b5a38a24911b => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:41.551353 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:41.632965 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:41.673893 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd828-766b-4767-935d-eec4b9096d96 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:41.707919 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:41.708820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:41.709103 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:41.709379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] PATH_INFO: `/v3/projects/faa43f27456748e49b23b5a38a24911b/users/11087bbc573b4699a3913d975359db41/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:41.711314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=faa43f27456748e49b23b5a38a24911b, user_id=11087bbc573b4699a3913d975359db41, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:41.712888 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba7092-9402-42ee-b060-2034682c3f7b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:41.728302 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 594/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:44:41 2022] PUT /identity/v3/projects/faa43f27456748e49b23b5a38a24911b/users/11087bbc573b4699a3913d975359db41/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:44:44.026065 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1abb3-3666-42e2-b06c-0119a653941e None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.026586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1abb3-3666-42e2-b06c-0119a653941e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.026586 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1abb3-3666-42e2-b06c-0119a653941e None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.028756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 599/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:44:44.034410 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.034870 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.035224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.061520 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:44:44.062981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:44:44.141736 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 595/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:44:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:44:44.165842 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1abb3-3666-42e2-b06c-0119a653941e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:44.258668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:44.280703 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1abb3-3666-42e2-b06c-0119a653941e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:44.315016 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:44.315807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.316091 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.316367 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.330132 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:44:44.338342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:44:44.339569 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:44:44.345894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 600/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:44:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:44:44.347053 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:44.413580 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:44.459379 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d1314d-ca9a-439e-b3fb-a81d690e2c4e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:44.495109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:44.496709 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.496709 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.496963 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.497705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:44.498009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:44.499356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:44.507614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 596/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:44.513653 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:44.614399 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505de55-e7e8-4303-9e20-8c07166c0eed None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:44.649722 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:44.650591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.650868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.651140 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] PATH_INFO: `/v3/users/11087bbc573b4699a3913d975359db41` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.652342 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] RBAC: Authorizing `identity:get_user(user_id=11087bbc573b4699a3913d975359db41)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:44.653815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:44.655451 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 601/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:44:44 2022] GET /identity/v3/users/11087bbc573b4699a3913d975359db41 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:44.661264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:44.762321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbe2b4-1d41-4322-a4f8-d8154b99f958 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:44.785132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:44.797471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:44.798123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:44.798402 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:44.798674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:44.866845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:44.868298 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:44.969269 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 597/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:44:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:44:44.974887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:44.986346 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:45.077735 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590517-8fb5-463f-b1d0-d7c0ec8596f5 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:45.112118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:45.113285 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:45.113562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:45.113832 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] PATH_INFO: `/v3/projects/faa43f27456748e49b23b5a38a24911b` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:45.115031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] RBAC: Authorizing `identity:get_project(project_id=faa43f27456748e49b23b5a38a24911b)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:45.116558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928fc4-243f-445d-af95-3acf384895d4 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:45.118263 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 602/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:44:44 2022] GET /identity/v3/projects/faa43f27456748e49b23b5a38a24911b => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:44:45.124611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:44:45.149559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:45.229514 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f96f3b-011d-446d-88f4-8135e16241d6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:44:45.272485 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:44:45.273145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:44:45.273420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:44:45.273688 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:44:45.274516 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=faa43f27456748e49b23b5a38a24911b, user.id=11087bbc573b4699a3913d975359db41) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:44:45.274826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:44:45.276255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f58d7-0da8-481e-946a-94beb2e2344d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:44:45.283176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 598/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:44:45 2022] GET /identity/v3/role_assignments?user.id=11087bbc573b4699a3913d975359db41&scope.project.id=faa43f27456748e49b23b5a38a24911b&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:44:55.107671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=62 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:44:55.280616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=56 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:48.321547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f05520-bf1c-489c-b7f0-4206bca42ea7 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.322096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f05520-bf1c-489c-b7f0-4206bca42ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.322096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f05520-bf1c-489c-b7f0-4206bca42ea7 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.323442 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 603/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:48.330151 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.330683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.330683 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.354821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:48.356250 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:48.455894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 599/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:45:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:45:48.480144 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f05520-bf1c-489c-b7f0-4206bca42ea7 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:48.614631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f05520-bf1c-489c-b7f0-4206bca42ea7 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:48.674027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:48.674964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.675250 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.675526 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.687073 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:48.688677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:48.708146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:48.709823 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:48.710891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:48.710891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:48.710891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:48.710891 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:45:48.712797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 604/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:48.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:48.854560 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71219-bbd7-4917-adbf-03c4a35a1ec1 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:48.896532 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:48.903995 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:48.904952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.905218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.905511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.906264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:48.906561 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:48.907920 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5c25-ff6c-452c-a6d6-08a534d767ed None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:48.928029 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 600/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:48.935136 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.005051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.066602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19773c0-8ffe-4eb5-a645-1c588ca09bca None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.113086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.113769 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.114054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.114330 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.115206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:49.116884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13a613-9a99-4b97-bfae-985305f395e6 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:49.142590 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 605/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:45:48 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:45:51.593566 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2661e5-f7f6-4c9c-bf7c-afc24bc64ec5 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.596270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2661e5-f7f6-4c9c-bf7c-afc24bc64ec5 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.596270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2661e5-f7f6-4c9c-bf7c-afc24bc64ec5 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.596270 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 601/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:51.601954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.602447 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.602575 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.621987 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:51.623216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:51.720466 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 606/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:45:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:45:51.745774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2661e5-f7f6-4c9c-bf7c-afc24bc64ec5 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.881511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2661e5-f7f6-4c9c-bf7c-afc24bc64ec5 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.930131 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.931005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.931381 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.931661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.945931 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:51.953823 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:51.954889 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:45:51.957791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:51.960534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 602/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:51.964438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.013805 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.096624 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c444c-7160-4cac-a56e-ffab1e5da250 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.141280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.142314 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.142744 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.143181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.148378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:52.148378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.148378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.163007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 607/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:52.170848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.244600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.303280 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1259e-0afe-45a2-b92b-dcc979073bb2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.362699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.363617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.364003 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.364377 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.377158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.378850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:52.401589 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:52.402774 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:52.403972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:52.403972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:52.403972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:52.403972 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:52.407907 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 603/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:52.415262 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.549453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.553449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9820a64-129f-4437-8f3b-f0f6d5d2edb3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.610415 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.611193 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.611551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.611901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.612745 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:52.613133 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.614624 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.629741 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 608/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:52.636747 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.779656 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde571b-20fc-4725-a656-170eb418604a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.781640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.836794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.837574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.837851 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.838114 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.848937 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.850404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:45:52.867435 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:52.868652 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:52.869727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 604/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:52.875020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.996247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d3cca-631b-4bf7-8561-3497d6aa599e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:53.035315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:53.036054 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:53.036347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:53.036712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:53.037929 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:53.038379 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:53.040254 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:53.063757 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 609/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:45:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:53.065076 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:53.185172 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5d426-71b1-4048-a9ef-2a5f97eb2f4a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:53.225835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:53.226488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:53.226768 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:53.227045 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:53.267899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:53.314623 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:53.316339 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:53.444385 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 605/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:45:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:45:53.446456 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:53.469228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:53.578791 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc5cad-322a-4bba-8200-b909a9848e5a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:53.621134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:53.622141 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:53.622514 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:53.622933 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:53.637260 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:53.639096 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:53.667835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:53.669034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:53.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:53.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:53.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:53.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:45:53.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 610/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:53.675185 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:53.762699 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:53.819300 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c14d04-08e7-4eba-9191-14d1b3babb08 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:53.860118 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:53.861001 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:53.861286 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:53.861585 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:53.862356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:53.862661 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:53.864168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:53.878568 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 606/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:53.885006 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:53.993239 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.038553 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b853842d-994e-4dec-ab0b-fe0aff0e8113 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.104679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.105515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.105875 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.106157 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.116449 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:54.116661 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:45:54.132488 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:54.133960 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:45:54.135496 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:45:54.138768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 611/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:54.140083 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.267422 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95575a01-a167-4f27-9478-d454cd3ac1f6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.295493 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.310370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.311118 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.311462 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.311796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.312696 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:54.313008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:54.314635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3144ac-2df3-4bd7-93bc-cb93733be90e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:54.330034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 607/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:54.336824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.472850 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21605ab8-5c66-44b3-a7ae-7c027aa63b28 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.521778 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.529011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.529809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.530093 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.530355 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.531265 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=57b9a2d97eff4d8aad78c189df251350) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:54.531580 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:54.533079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c769c2-e6ce-4f2f-a24f-c09b56b1b02b None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:54.548446 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 612/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/role_assignments?user.id=57b9a2d97eff4d8aad78c189df251350&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:57.036021 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fa7e-eaa5-43fc-82fa-f8a3910094fa None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.036794 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fa7e-eaa5-43fc-82fa-f8a3910094fa None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.037278 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48fa7e-eaa5-43fc-82fa-f8a3910094fa None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.040052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 608/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:45:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:57.046730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.046730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.047908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.073317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:57.074560 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:57.154919 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 613/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:45:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:45:57.182821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fa7e-eaa5-43fc-82fa-f8a3910094fa None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.313212 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48fa7e-eaa5-43fc-82fa-f8a3910094fa None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.355721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.356565 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.356850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.357141 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.371089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:57.379341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:57.380634 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:57.381624 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 609/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:57.387326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.534952 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f053c3c-9a35-4403-9463-c8645a0f05bc None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.560326 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.577970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.578634 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.578924 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.579205 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.580013 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] RBAC: Adding query filter params (name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:57.580327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.581803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.591949 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 614/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:57.598575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.660339 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.741243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069faee-57c5-47bc-b77f-1ab5692e83cc None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.782010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.782833 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.783158 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.783416 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.797636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.799143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:57.818743 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:57.819825 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:57.821404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:57.821404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:57.821404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:57.821404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:57.821404 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 610/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:57.826197 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4f527-573b-45bc-8d1b-a07f5ab97964 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.007060 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.007726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.008592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.008852 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.009705 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:58.010014 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.011602 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.026430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 615/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.032819 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.144953 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.158291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317baa67-bba0-41d3-8cc6-a41f192e5b01 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.199998 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.201027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.201439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.201820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.211065 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.213423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:45:58.230027 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:58.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:58.233674 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 611/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:58.237679 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.292512 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.355596 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8a02-2115-4750-b1ff-7b8aa2abc67f None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.398311 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.399557 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.399884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.400180 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.401128 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:58.401434 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.402991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.425428 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 616/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.431370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.551260 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d883b13-1bf9-41af-a3f5-4f949be6d366 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.590820 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.591721 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.591903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.592171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.606745 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.608642 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:58.632508 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:58.633705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:58.634916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:58.634916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:58.634916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:58.634916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:58.634916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 612/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:58.640591 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.759950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d7e6d-88e6-4fb1-a548-ed4bd2e96b49 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.762417 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.798983 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.799648 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.799930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.800202 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.800958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:58.801317 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.803843 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.816143 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 617/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.823488 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.835922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.947368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d98ed-8e2e-4b35-a11f-84cce826b6f2 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.989126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.990010 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.990289 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.990559 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.998094 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.999586 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:45:59.016005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:59.017056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:45:59.017952 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 613/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:59.023508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:59.141334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8bb68-8343-4105-b54a-64e5b1650648 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:59.181870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:59.182636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:59.182983 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:59.183280 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:59.184116 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:59.184481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:59.186043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac7b37d-ec85-4a96-807c-e58bc890806f None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:59.201674 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 618/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:59.207784 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:59.282977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:59.326431 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef239ee1-7571-4ad2-be9b-cdb09ebc6add None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:59.365627 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:59.366606 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] REQUEST_METHOD: `PUT` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:59.366928 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:59.367218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/57b9a2d97eff4d8aad78c189df251350/roles/7d35c178c429464cae2e3fe1d26c4ac2` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:59.385756 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=57b9a2d97eff4d8aad78c189df251350, role_id=7d35c178c429464cae2e3fe1d26c4ac2)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:59.387530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66570834-cd46-4713-b2a7-64eb4bdbc883 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:59.411173 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 614/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:45:59 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/57b9a2d97eff4d8aad78c189df251350/roles/7d35c178c429464cae2e3fe1d26c4ac2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:01.845562 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916f680-728b-4e53-a9b2-f7c49b2163de None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.846196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916f680-728b-4e53-a9b2-f7c49b2163de None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.846196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916f680-728b-4e53-a9b2-f7c49b2163de None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.847579 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 619/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:01.854356 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.854921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.854921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.874374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:01.875993 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:01.976632 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 615/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:02.001773 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916f680-728b-4e53-a9b2-f7c49b2163de None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.028686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.120082 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916f680-728b-4e53-a9b2-f7c49b2163de None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.161106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.162020 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.162333 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.162671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.176191 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:02.184827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:02.185992 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:02.186848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:02.191334 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 620/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:02.192512 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.247458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.317143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c8a92-64b9-42c3-a926-99c138585d0e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.357358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.358091 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.358380 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.358679 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.359537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] RBAC: Adding query filter params (name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:02.359835 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.361342 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.373903 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 616/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:02.379499 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.476565 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.496376 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b300f4-6988-4194-88a7-a4e33553953e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.536635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.537439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.537740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.537990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.551582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.553420 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:02.575539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:02.576738 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:02.577695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:02.577695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:02.577695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:02.577695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:02.577695 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 621/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:02.582727 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.711750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce6d4-5885-45d2-bee7-39504db0c24b None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.725142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.755490 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.756264 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.756675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.756991 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.757843 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:02.758152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.759765 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.773362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 617/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:02.780075 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.895861 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ec259-949d-46e8-afbc-341f0ea46c58 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.941227 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.942298 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.942610 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.942928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.948749 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.955974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.957821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:02.975614 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:02.976837 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:02.977884 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 622/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:02.986131 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.139579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b7599-06b0-4128-9c27-43bbdebda20e None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.178291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.178973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.179292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.179598 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.180608 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:03.180921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.183198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.205564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 618/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:03.211465 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.346792 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7126c-f31b-4c65-b088-5709db326ba3 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.389476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.390113 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.390397 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.390671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.464118 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.471295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.472815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.590270 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 623/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:46:03.596535 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.631729 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.723165 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f93c0-8a78-4e87-88d1-728796808de5 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.761120 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.761975 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.762305 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.762558 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.773947 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.775433 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:03.796358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:03.797476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:03.798347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:03.798347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:03.798347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:03.798347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:03.798347 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 619/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:03.803633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.917799 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.931066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21063a2-3835-4f0b-8943-3b10795ddb5c None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.971519 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.971697 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.971978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.972328 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.973035 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:03.973352 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.974771 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.990101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 624/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:03.996108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.098938 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:04.129129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894407b-4488-4d6b-b08a-8041ae4824e7 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.168692 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.169418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.169720 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.169966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.177150 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.178597 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:04.194315 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:04.195348 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:04.196458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 620/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:04.201866 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.318986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788dd8-9f66-4df6-934b-50c2f719d7d0 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.363427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.364086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.364386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.364698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.365737 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.366045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.367553 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68193123-f796-43e4-857c-b0040114d7b1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.382297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 625/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.389073 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.515909 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2538d39-e591-43cc-a4a9-f7c63481a9a6 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.567518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.568263 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.568583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.568865 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.569832 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] RBAC: Adding query filter params (role.id=7d35c178c429464cae2e3fe1d26c4ac2, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=57b9a2d97eff4d8aad78c189df251350) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.570175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.571659 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43573e6-541f-452f-ad2d-2f0ce9a5d733 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.583705 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 621/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/role_assignments?user.id=57b9a2d97eff4d8aad78c189df251350&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=7d35c178c429464cae2e3fe1d26c4ac2 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.144229 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b1f0-d301-4c8e-995f-664d6b765282 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.144851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b1f0-d301-4c8e-995f-664d6b765282 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.144851 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934b1f0-d301-4c8e-995f-664d6b765282 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.146176 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 626/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:07.152171 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.153271 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.153387 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.159303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.173961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:07.175321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:07.251208 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 622/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:46:07.276291 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b1f0-d301-4c8e-995f-664d6b765282 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.296347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.427509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934b1f0-d301-4c8e-995f-664d6b765282 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.469906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.470726 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.471009 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.471284 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.483848 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:07.493056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:07.494252 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:07.495613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 627/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:07.502259 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.635635 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8bd3-0ec0-4cdf-88b2-8fb2eca042ad None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.688918 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.695124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.695826 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.696203 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.696514 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.697414 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:07.697790 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.699418 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.712439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 623/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.716452 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.807402 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.833248 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e74d41-5de0-4b1c-95e9-002d014e053a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.874774 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.875706 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.875997 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.876272 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.890253 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.892108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:07.911916 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:07.913232 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:07.914318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:07.914318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:07.914318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:07.914318 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:07.916717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 628/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:07.919449 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.047882 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bb777-679b-4472-bbfe-f84252ccb8fe None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.085921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.086511 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.086791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.087064 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.088848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:08.088848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.089489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.113040 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 624/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:08.119783 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.233548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5e4ee4-d8ee-4cf0-bb3f-80807a071a8a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.269755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.278789 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.279598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.279877 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.280153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.288156 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.289815 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:08.304803 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:08.311812 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:08.313114 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 629/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:08.315016 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.397574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.450354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19b8fb-0007-4856-b87a-c3981cd489f1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.496646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.497845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.498126 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.498398 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.499253 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:08.499567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.501051 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.521977 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 625/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:08.528199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.642550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40504e15-b5c1-4db7-9f7a-99fddf0bb66d None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.690389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.690897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.691182 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.691460 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.754426 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.769386 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.770882 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.891374 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 630/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:46:08.897391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.015691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edac8a-4025-4ffc-88e2-623008e96e7a None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:09.040368 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:09.063649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:09.064464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.064796 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.065025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.074942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:09.076741 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:09.096673 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:09.097767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:09.098694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:09.098694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:09.098694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:09.098694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:09.098694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 626/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:09.103703 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.194241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:09.207718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110154-0ffe-474e-a13c-d17941b82e3a None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:09.242793 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:09.243444 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.243644 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.243920 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.244687 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:09.245000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:09.246406 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:09.259439 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 631/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:09.265152 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.373318 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41610303-e7a7-475f-b845-2662a2ec06da None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:09.412314 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:09.413142 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.413424 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.413701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.421052 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:09.422506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:09.436875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:09.437894 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:09.439108 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 627/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:09.444028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.545169 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ebd1c-3052-4d6d-92dc-56c64c014dfe None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:09.579846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:09.580542 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.580826 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.581101 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] PATH_INFO: `/v3/projects` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.581887 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:09.582196 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:09.583635 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c231a8-f927-4795-8d45-93b338bcebac None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:09.595718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 632/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:09.602006 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.629458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:09.724341 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fbf6-928a-4a0f-ba44-fb0c3f1410e1 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:09.757933 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:09.758592 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.758867 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.759137 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.759980 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=57b9a2d97eff4d8aad78c189df251350) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:09.760290 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:09.761763 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c280f55-8768-4226-b766-018f8f144a9f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:09.768015 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 628/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:46:09 2022] GET /identity/v3/role_assignments?user.id=57b9a2d97eff4d8aad78c189df251350&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:12.055040 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5aae3b-3a7c-4b33-ae66-2f88746d3f0a None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.055604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5aae3b-3a7c-4b33-ae66-2f88746d3f0a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.055604 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5aae3b-3a7c-4b33-ae66-2f88746d3f0a None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.056978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 633/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:12.063283 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.063715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.063839 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.081352 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:12.082579 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:12.161578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 629/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:46:12.184671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5aae3b-3a7c-4b33-ae66-2f88746d3f0a None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.195677 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.301349 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5aae3b-3a7c-4b33-ae66-2f88746d3f0a None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.337481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.338271 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.338551 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.338824 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.349847 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:12.357592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:12.358700 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:12.359671 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 634/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:12.365239 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.383821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.465829 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcba8e1-fae8-4df5-a298-cd22810da4ce None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.502374 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.503030 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.503313 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.503588 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] PATH_INFO: `/v3/roles` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.504332 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:12.504669 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.506033 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.514017 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 630/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:12.520295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.588828 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.624964 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e64466-737d-4362-9e8d-7b177aaa1664 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.665310 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.666151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.666481 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.666788 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.682525 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:12.705867 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:12.707028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:12.707948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:12.707948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:12.707948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:12.707948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:12.707948 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 635/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:12.713281 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.819396 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c249d-b7f4-484e-a749-270332d5f258 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.853764 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.854333 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.854610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.854881 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.855600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:12.855902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.857279 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.870071 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 631/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:12.875808 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.982668 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acf51a-a96f-49a9-b698-809cdc9c2ebd None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.008487 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:13.023159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.023956 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.024237 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.024539 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.032894 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.034351 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:13.056979 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:13.058611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:13.059630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 636/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:13.064229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:13.166574 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0421ddfb-09c1-43ac-b13d-d2474910f03c None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.200864 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.202159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.202563 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.202845 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] PATH_INFO: `/v3/users` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.203704 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:13.204020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.205536 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.215092 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:13.225393 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 632/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:46:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:13.231027 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:13.332794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578fdf22-26a7-4a74-93b5-b7f0be638e07 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.367560 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.368365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.368930 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.369208 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.380616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.382072 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:13.399490 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:13.400582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:13.401489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:13.401489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:13.401489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:13.401489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:13.401489 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 637/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:13.406650 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:13.510610 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69b017-94a1-4f28-86fa-2e2f713e4271 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.545571 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.546224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.546537 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.546782 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.547499 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:13.547803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.549215 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.560227 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 633/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:13.566628 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:13.600145 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:13.684302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa273cf-1eb3-4968-a2f1-27c447505725 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.719224 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.720011 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.720292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.720597 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.736360 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.737807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:13.753719 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:13.754776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:13.755713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 638/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:13.760840 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:13.821935 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:13.874926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848f911-53f2-4cd9-a904-a1b4da09311f None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:13.910827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:13.911482 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.911766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.912041 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.912846 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:13.913159 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:13.914578 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f6696-a2f4-46c7-bbef-96231f6956cf None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:13.927335 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 634/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:13.933302 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.027247 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.037106 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d6f5e-5f03-4985-8506-ea119ea54a90 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.073068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.073953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] REQUEST_METHOD: `PUT` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.074231 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.074503 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] PATH_INFO: `/v3/projects/da6a49f3e751495596447ab1d795277f/users/57b9a2d97eff4d8aad78c189df251350/roles/6abc0b064ff0442cbb5ffabd46894792` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.076476 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] RBAC: Authorizing `identity:create_grant(project_id=da6a49f3e751495596447ab1d795277f, user_id=57b9a2d97eff4d8aad78c189df251350, role_id=6abc0b064ff0442cbb5ffabd46894792)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:14.077954 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fec3ff-3020-4f9c-9574-e9e4825ed343 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:14.097043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 639/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:46:13 2022] PUT /identity/v3/projects/da6a49f3e751495596447ab1d795277f/users/57b9a2d97eff4d8aad78c189df251350/roles/6abc0b064ff0442cbb5ffabd46894792 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:46:16.325391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4890c-9d9a-48d6-b547-e4366b0685fe None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.325904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4890c-9d9a-48d6-b547-e4366b0685fe None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.325904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4890c-9d9a-48d6-b547-e4366b0685fe None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.327217 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 635/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:16.333689 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.334245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.334245 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.351435 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:16.352641 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:16.419807 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 640/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:16.444275 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4890c-9d9a-48d6-b547-e4366b0685fe None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.551837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4890c-9d9a-48d6-b547-e4366b0685fe None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.569878 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.587802 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.588564 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.588837 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.589116 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.601451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:16.609456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:16.610524 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:16.611758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 636/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:16.616725 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.713056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.733287 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb462f3-a5a4-419d-9094-886cfc83220e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.775199 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.775864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.776148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.776445 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] PATH_INFO: `/v3/roles` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.777200 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:16.777509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.782768 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.799633 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 641/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:16.805816 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.922904 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc3821-ab72-467e-b22e-409031bab271 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.960288 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.961122 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.961401 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.961675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.973420 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.974866 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:16.992543 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:16.993619 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:16.999834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:16.999834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:16.999834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:16.999834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:16.999834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 637/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:17.000779 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.102361 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14458a-bf22-4139-a99f-38dad165aa4e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.137370 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.138029 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.138309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.138581 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.139293 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:17.139599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.141103 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.153430 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 642/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:17.159664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.192020 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.264817 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c312bc-d3b1-4fc2-89f5-b6e01a7058af None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.300229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.301056 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.301338 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.301611 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.309046 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.310495 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:17.324811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:17.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:17.327628 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 638/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:17.332495 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.336327 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.444730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e2091a-d25b-44ba-884f-971840dffeb7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.478636 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.479315 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.479592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.479865 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] PATH_INFO: `/v3/users` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.480736 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:17.481046 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.482545 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.505686 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 643/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:17.511762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.611948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.622211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992c3fb-616d-4728-b2e0-ba8d25eea642 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.664719 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.665363 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.665644 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.665916 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.739636 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.741069 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.837109 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.848983 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 639/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:46:17.855177 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.918537 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.965825 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f195-75a0-49fc-b58e-033b360c8095 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.999902 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.001195 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.001472 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.001739 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.013022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.014505 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:18.033000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:18.034056 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:18.034974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:18.034974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:18.034974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:18.034974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:18.034974 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 644/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:18.039966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.142290 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43393b68-72d4-4dbb-a20a-2ec5169f539d None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.181481 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.182138 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.182416 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.182686 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] PATH_INFO: `/v3/domains` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.183405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.183712 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.185090 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.196961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 640/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:18.202582 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.304999 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03f93e-0433-44ea-8e5e-ec2cbe5ed7ea None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.320135 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.341147 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.341928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.342206 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.342473 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.349062 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.350528 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:18.365797 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application with Lock( May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:18.366899 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: ERROR keystone.server.flask.application May 19 19:46:18.367986 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 645/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:18.373617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.433715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.487787 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb14cda-41cb-4d9b-83a2-bc0e761cc1cc None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.521966 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.522614 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.522899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.523175 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] PATH_INFO: `/v3/projects` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.523957 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.524267 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.525722 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b6e03-4ecb-4ee7-b5dc-dea24dd448d3 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.539447 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 641/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:18.545631 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.650300 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390f94d-da67-4af0-bba9-ab742fecb42e None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.685222 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.685872 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.686154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.686431 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.687297 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] RBAC: Adding query filter params (role.id=6abc0b064ff0442cbb5ffabd46894792, scope.project.id=da6a49f3e751495596447ab1d795277f, user.id=57b9a2d97eff4d8aad78c189df251350) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.687612 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.689146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffca90b-de57-49da-8f46-2c806733ef1c None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.695794 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 646/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/role_assignments?user.id=57b9a2d97eff4d8aad78c189df251350&scope.project.id=da6a49f3e751495596447ab1d795277f&role.id=6abc0b064ff0442cbb5ffabd46894792 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:21.029530 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b85b-31d5-4783-834e-5e2704ead64e None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.030055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b85b-31d5-4783-834e-5e2704ead64e None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.030055 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08b85b-31d5-4783-834e-5e2704ead64e None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.031458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 642/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:21.037556 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.038076 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.038160 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.054730 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:21.055928 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:21.086429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:21.125536 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 647/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:21.148951 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b85b-31d5-4783-834e-5e2704ead64e None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.254649 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08b85b-31d5-4783-834e-5e2704ead64e None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.265506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:21.288990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.289781 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.290061 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.290333 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.291008 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:21.292471 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:21.305487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:21.306767 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:21.307615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 643/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:21.313045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.425148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e04ec-93c6-49bd-a238-37a2a143e123 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.469000 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.469666 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.470023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.470946 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.471990 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:21.472301 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:21.473713 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c25b1-098d-4b38-864e-2c9d892a9f55 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:21.478718 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 648/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:21.484575 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.587243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed040874-e60b-41d1-98e2-1265fda41167 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.631766 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.632450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.632737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.633014 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] PATH_INFO: `/v3/services` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.633718 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] RBAC: Adding query filter params (type=placement) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:21.634026 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:21.635391 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428f150-8a98-4c18-9139-1b27613cb9bb None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:21.640327 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 644/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:23.864571 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364eae9-bf3d-49b9-a43e-0942f989ce9c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.865174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364eae9-bf3d-49b9-a43e-0942f989ce9c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.865174 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364eae9-bf3d-49b9-a43e-0942f989ce9c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.866469 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 649/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:23.871926 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee7c0-16a9-40e0-8860-a4896bdcd680 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.872358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee7c0-16a9-40e0-8860-a4896bdcd680 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.872769 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee7c0-16a9-40e0-8860-a4896bdcd680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.889877 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c46ee7c0-16a9-40e0-8860-a4896bdcd680 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:23.891059 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c46ee7c0-16a9-40e0-8860-a4896bdcd680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:23.922513 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:23.961034 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 645/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:23.985442 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364eae9-bf3d-49b9-a43e-0942f989ce9c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:24.018117 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:24.115307 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364eae9-bf3d-49b9-a43e-0942f989ce9c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:24.150329 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:24.150982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.151258 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.151525 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.152241 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:24.153644 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32032237-5c17-4e79-a5ac-4a7f99c7c6ad None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:24.167967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 650/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:46:23 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:46:26.473198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0a3d-8719-4f9e-bb04-ee0d8137a359 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.473762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0a3d-8719-4f9e-bb04-ee0d8137a359 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.473762 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be0a3d-8719-4f9e-bb04-ee0d8137a359 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.475100 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 646/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:26.480207 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.480603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.480800 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.497747 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:26.499242 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:26.573871 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 651/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:26.596358 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be0a3d-8719-4f9e-bb04-ee0d8137a359 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:26.724476 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be0a3d-8719-4f9e-bb04-ee0d8137a359 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:26.758865 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:26.759670 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.759953 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.760229 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.760948 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:26.762360 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:26.775646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:26.776887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:26.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 647/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:46:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:26.783119 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:26.801840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:26.890081 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb102240-993e-43eb-b52b-82fc640f2297 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:26.924740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:26.925474 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.925752 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.926023 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.926723 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:26.927026 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:26.928530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.933698 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 652/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 19:46:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:26.939600 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:26.972994 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:27.044459 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc9114-83aa-4b42-ac01-18a7fee687d0 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:27.077701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:27.078362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:27.078639 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:27.078910 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:27.079703 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] RBAC: Adding query filter params (interface=public, service_id=8c5bfd200cc34a24b22ce4caae76c0a5, region_id=RegionOne) {{(pid=86704) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:27.080012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:27.082042 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1272a-9888-43e3-a13e-bd69dc3b74f8 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:27.087506 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 648/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:46:26 2022] GET /identity/v3/endpoints?service_id=8c5bfd200cc34a24b22ce4caae76c0a5&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:27.093158 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:27.183853 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:27.202811 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40626f-581e-421a-b38e-3b5530fd27a7 None admin] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:27.238070 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:27.238729 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:27.239007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:27.239276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:27.239958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:27.241353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed757b-df35-4b1c-9b0c-6527f85a0890 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:27.248115 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 653/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:46:27 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:46:29.478861 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a89dcd-43d5-4c92-a753-9446478bbabf None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.479399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a89dcd-43d5-4c92-a753-9446478bbabf None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.479399 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a89dcd-43d5-4c92-a753-9446478bbabf None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.480811 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 649/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:29.485955 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.486335 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.486508 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.503809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:29.504980 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:29.572626 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 654/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:46:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:46:29.596050 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a89dcd-43d5-4c92-a753-9446478bbabf None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.699730 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.707480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a89dcd-43d5-4c92-a753-9446478bbabf None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.746211 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.746922 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.747207 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.747487 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.748143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:29.749809 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:29.761944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:29.763154 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application with Lock( May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: ERROR keystone.server.flask.application May 19 19:46:29.764025 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 650/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:29.769413 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.847471 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.871151 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb53414-7b45-4e87-bc2d-d3683ac34f0e None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.906890 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.907552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.907829 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.908102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] PATH_INFO: `/v3/services` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.908827 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:29.909134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:29.910746 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa162960-0db6-4e90-b321-5fd362107dd1 None admin] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:29.915801 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 655/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:29.921582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.031527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1234fc-972e-444e-a9c5-47ea81b31255 None admin] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.065538 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8197d33213ab455786d2c8bd2697b7ff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.066190 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.067009 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.067241 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.067979 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.069362 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170b19a-a2c6-4652-9ab2-5074cde8f7b7 None admin] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.089454 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 651/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:46:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:39.912780 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=30 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:40.085664 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.178592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d87eb8-a8f9-4fe5-8b2f-cf6fd92f0ea5 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.179161 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d87eb8-a8f9-4fe5-8b2f-cf6fd92f0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.179161 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d87eb8-a8f9-4fe5-8b2f-cf6fd92f0ea5 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.184989 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 656/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:36.186687 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d92fd-6a7e-42fc-b057-7c0d27631b4f None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.187287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d92fd-6a7e-42fc-b057-7c0d27631b4f None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.187287 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d92fd-6a7e-42fc-b057-7c0d27631b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.279179 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-d49d92fd-6a7e-42fc-b057-7c0d27631b4f None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:36.280423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-d49d92fd-6a7e-42fc-b057-7c0d27631b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:36.420423 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 652/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:47:36.461425 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.462073 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.462550 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.464721 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 657/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 19:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:36.471168 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dfa2-babf-40a6-a6ff-421ab85b3f22 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.471465 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dfa2-babf-40a6-a6ff-421ab85b3f22 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.471747 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277dfa2-babf-40a6-a6ff-421ab85b3f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.560640 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-a277dfa2-babf-40a6-a6ff-421ab85b3f22 None None] MFA Rules not processed for user `e56d824c48c74091a66af4d24459c046`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:36.561973 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-a277dfa2-babf-40a6-a6ff-421ab85b3f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:36.585256 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.672439 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 653/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 19:47:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:36.681755 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.684008 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:36.836121 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f884e866-6992-41a4-8859-57b9eb02ec1c None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.924261 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.925051 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.925398 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.925740 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.927840 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:36.994813 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:37.058134 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:37.059603 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124fe5c2-d551-4ef7-ae40-6444afb5fca8 service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:37.080365 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 658/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:39.670827 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.671366 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.671657 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.673303 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 654/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:39.679340 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87667-7b1a-4e13-ade5-f194ab5bee6e None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.679885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87667-7b1a-4e13-ade5-f194ab5bee6e None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.679885 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f87667-7b1a-4e13-ade5-f194ab5bee6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.759068 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c6f87667-7b1a-4e13-ade5-f194ab5bee6e None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:39.760276 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c6f87667-7b1a-4e13-ade5-f194ab5bee6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:39.865908 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 659/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:47:39.904548 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.906124 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:40.037375 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f711bc-157c-45e2-89d4-dad77651e70d None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.118373 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.121436 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.122190 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.122961 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.123617 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.125897 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:40.265791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.269089 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a795601-3da0-4e20-85c0-29a44911c8e8 service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.292523 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 655/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:43.013403 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6894-4ca3-4a35-b9c1-78ffb0245005 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.013962 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6894-4ca3-4a35-b9c1-78ffb0245005 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.013962 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa6894-4ca3-4a35-b9c1-78ffb0245005 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.015344 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 660/1315] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:43.021376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d53b93-000b-4bac-848e-9baee537b91b None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.021924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d53b93-000b-4bac-848e-9baee537b91b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.022130 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d53b93-000b-4bac-848e-9baee537b91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.088132 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-16d53b93-000b-4bac-848e-9baee537b91b None None] MFA Rules not processed for user `a66e55122f6a41078aeed1852f492fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:43.089405 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-16d53b93-000b-4bac-848e-9baee537b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:43.143112 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:43.189291 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 656/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:43 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:43.242378 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.242670 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.242953 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.244337 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 661/1317] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:43.250224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7124-0b56-437e-81ce-1ecc209539a7 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.250518 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7124-0b56-437e-81ce-1ecc209539a7 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.250798 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac7124-0b56-437e-81ce-1ecc209539a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.328874 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-41ac7124-0b56-437e-81ce-1ecc209539a7 None None] MFA Rules not processed for user `e56d824c48c74091a66af4d24459c046`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:43.338539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-41ac7124-0b56-437e-81ce-1ecc209539a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:43.416525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 657/1318] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 19:47:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:43.425599 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:43.427146 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:43.445607 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:43.553906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eb1d0-e0bc-4342-b212-5cb78f6708c0 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:43.629077 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:43.629732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.630015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.630291 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.632125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:43.706080 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:43.764870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:43.766325 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8621f371-b9bb-4c1d-88db-7465973d0d29 service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:43.786087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 662/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:43 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:47.001261 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.001848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.001848 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.003188 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 658/1320] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:47.008756 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224321e4-ad45-4ebf-88dc-67f346fdc52c None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.009178 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224321e4-ad45-4ebf-88dc-67f346fdc52c None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.009309 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224321e4-ad45-4ebf-88dc-67f346fdc52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.075175 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-224321e4-ad45-4ebf-88dc-67f346fdc52c None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:47.076380 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-224321e4-ad45-4ebf-88dc-67f346fdc52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:47.174870 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 663/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:47.214715 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:47.216292 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:47.268479 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:47.332234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb4610-8321-460f-86a4-1a106ecac4ea None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:47.409531 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:47.410198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.410475 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.410744 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.412376 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:47.512354 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:47.527035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:47.528428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11ef18-94dd-400b-9bf5-4687b65a1f41 service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:47.546218 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 659/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:50.617914 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.618427 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.618509 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.619958 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 664/1323] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:50.626237 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8245e3-5b36-485b-97f8-44df9e382822 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.626911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8245e3-5b36-485b-97f8-44df9e382822 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.626911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8245e3-5b36-485b-97f8-44df9e382822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.711198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-be8245e3-5b36-485b-97f8-44df9e382822 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:50.712583 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-be8245e3-5b36-485b-97f8-44df9e382822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:50.819529 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 660/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:50.865448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:50.867007 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:50.887274 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:50.985982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7d18f-8fcc-4574-a224-ded898d3b676 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:51.075074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:51.075727 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.076015 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.076292 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.078028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:51.166025 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:51.207109 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:51.209959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d05463-298b-49c1-839e-1be3804f29a7 service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:51.232532 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 665/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:53.630615 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:53.631149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:53.631149 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:53.633378 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 661/1326] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:53.638862 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4d2a-e5d5-4493-a651-07f9e130c976 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:53.639357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4d2a-e5d5-4493-a651-07f9e130c976 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:53.639462 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b4d2a-e5d5-4493-a651-07f9e130c976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:53.711877 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-1e2b4d2a-e5d5-4493-a651-07f9e130c976 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:53.713156 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-1e2b4d2a-e5d5-4493-a651-07f9e130c976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:53.813775 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 666/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:53.849901 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:53.851646 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:53.925515 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:53.980480 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f913d-2bb3-4759-86e3-661469d30fd7 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:54.057086 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:54.057733 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.058005 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.058284 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.059924 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:54.187182 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:54.190539 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:54.192156 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2889df-5aed-4e4c-adbb-aebe9b8ad560 service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:54.211293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 662/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:56.553079 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.553620 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.553620 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.555214 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 667/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:56.560944 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1e03-00f0-4479-964b-06a9e2e7ef09 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.561403 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1e03-00f0-4479-964b-06a9e2e7ef09 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.561594 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1e03-00f0-4479-964b-06a9e2e7ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.640694 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-1ddb1e03-00f0-4479-964b-06a9e2e7ef09 None None] MFA Rules not processed for user `a66e55122f6a41078aeed1852f492fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:56.641921 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-1ddb1e03-00f0-4479-964b-06a9e2e7ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:56.750551 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 663/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:56.795035 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.796675 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:56.829087 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.908448 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c40a8-c0aa-4d98-97e0-05d958203813 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.990878 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.991540 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.991821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.992102 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.994809 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:57.083336 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:57.127393 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:57.128938 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21abe85f-279d-44d6-9ed7-d776087aab3a service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:57.148226 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 668/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:47:59.710875 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.711483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.711483 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.713088 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 664/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:59.719154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da0c1-7434-4b80-a6fa-8c567d343922 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.719715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da0c1-7434-4b80-a6fa-8c567d343922 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.719715 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da0c1-7434-4b80-a6fa-8c567d343922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.789513 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-960da0c1-7434-4b80-a6fa-8c567d343922 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:59.790712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-960da0c1-7434-4b80-a6fa-8c567d343922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:59.904515 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 669/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:47:59.940556 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.942131 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:00.016489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.053067 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae3050-bb9f-4e81-beed-b1ef65e4fbe5 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.131187 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.131747 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.132024 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.132293 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.134681 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:00.268243 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.269752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aee81e-918e-42ee-a986-ff5e2c207f1e service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.282428 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.293409 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 665/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:02.626438 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031c52-c923-4996-82fd-862363fa721f None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:02.626957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031c52-c923-4996-82fd-862363fa721f None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:02.626957 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b031c52-c923-4996-82fd-862363fa721f None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:02.628283 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 670/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:02.634365 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec387-5dc3-4081-afa3-d9cc392acfaf None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:02.635110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec387-5dc3-4081-afa3-d9cc392acfaf None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:02.635110 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec387-5dc3-4081-afa3-d9cc392acfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:02.712370 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-178ec387-5dc3-4081-afa3-d9cc392acfaf None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:02.713641 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-178ec387-5dc3-4081-afa3-d9cc392acfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:02.818717 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 666/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:02.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031c52-c923-4996-82fd-862363fa721f None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:02.866901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-7b031c52-c923-4996-82fd-862363fa721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:02.917588 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:02.984798 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b031c52-c923-4996-82fd-862363fa721f None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.067611 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.068264 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.068579 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.068855 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.070523 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:03.156627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:03.184048 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:03.185455 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73266-b3ab-416c-8912-877026ccd673 service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:03.203978 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 671/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:05.948294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.948990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.948990 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.950269 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 667/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:05.955548 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6c2a-4df0-480e-9037-8803453e2b93 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.955982 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6c2a-4df0-480e-9037-8803453e2b93 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.956108 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b6c2a-4df0-480e-9037-8803453e2b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.034181 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-642b6c2a-4df0-480e-9037-8803453e2b93 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:06.035421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-642b6c2a-4df0-480e-9037-8803453e2b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:06.157552 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 672/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:06.198795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.201791 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:06.285642 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:06.318255 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44530c-486c-44b2-873d-859a41e71fee None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.330805 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.331456 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.331737 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.332012 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.333752 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:06.451245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:06.452625 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77a8f9-dff7-4b2c-84da-e7b6a786ba94 service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:06.473542 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 668/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:10.999031 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:10.999555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:10.999555 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:11.000950 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 673/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:11.006396 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399bb81-e434-4022-994a-dd01ac51b14a None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:11.006806 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399bb81-e434-4022-994a-dd01ac51b14a None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:11.006964 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c399bb81-e434-4022-994a-dd01ac51b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:11.056987 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-c399bb81-e434-4022-994a-dd01ac51b14a None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:11.057967 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-c399bb81-e434-4022-994a-dd01ac51b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:11.114803 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=100 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:11.151224 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 669/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:11.190554 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:11.192835 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:11.217598 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6f22c-1396-4da1-9303-85997cc0650d None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:11.221228 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:11.221868 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:11.222153 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:11.222429 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:11.235506 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:11.237043 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20886085-722e-445f-86c4-00238bfb3d0b service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:11.247717 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 674/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:14.175758 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:14.176294 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:14.176527 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:14.177959 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 670/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:14.183991 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88766-e421-4bf6-9393-be3af3197a08 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:14.184441 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88766-e421-4bf6-9393-be3af3197a08 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:14.184693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88766-e421-4bf6-9393-be3af3197a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:14.216159 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-fab88766-e421-4bf6-9393-be3af3197a08 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:14.217453 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-fab88766-e421-4bf6-9393-be3af3197a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:14.226041 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 675/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:14.272198 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:14.274037 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:14.287895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48e6c3-0374-41c8-be68-40c929268ca7 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:14.291266 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:14.292386 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:14.292850 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:14.293129 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:14.302450 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:14.303887 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0929d3-762a-4482-89b0-dd648a8420ad service neutron] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:14.318775 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 671/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:18.881637 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a7d8-56d2-4231-9026-d9f73a866cf0 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.882230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a7d8-56d2-4231-9026-d9f73a866cf0 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.882230 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a7d8-56d2-4231-9026-d9f73a866cf0 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.883592 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 676/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:18.889451 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf750b0-bb6b-4846-a88a-22d8c773be69 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.889899 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf750b0-bb6b-4846-a88a-22d8c773be69 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.889998 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf750b0-bb6b-4846-a88a-22d8c773be69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.913337 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-1cf750b0-bb6b-4846-a88a-22d8c773be69 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:18.914549 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-1cf750b0-bb6b-4846-a88a-22d8c773be69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:18.922821 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 672/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:18.961421 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8a7d8-56d2-4231-9026-d9f73a866cf0 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:18.976485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8a7d8-56d2-4231-9026-d9f73a866cf0 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:18.980347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e56d824c48c74091a66af4d24459c046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:18.981494 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.981776 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.982052 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.988034 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:18.992606 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:18.994045 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7bfa-60ec-43c8-bdb5-b29c08110315 service neutron] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:19.005295 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 677/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:24.622458 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fc54-f2ab-4e7c-be54-123547447587 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:24.623082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fc54-f2ab-4e7c-be54-123547447587 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:24.623082 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13fc54-f2ab-4e7c-be54-123547447587 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:24.627145 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 673/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 19:48:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:24.630152 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162537f-999c-481a-9a20-8909791c91a5 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:24.630638 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162537f-999c-481a-9a20-8909791c91a5 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:24.630732 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162537f-999c-481a-9a20-8909791c91a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:24.724967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-d162537f-999c-481a-9a20-8909791c91a5 None None] MFA Rules not processed for user `57b9a2d97eff4d8aad78c189df251350`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:24.726821 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-d162537f-999c-481a-9a20-8909791c91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:24.825864 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 678/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 19:48:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:48:28.907498 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=19 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:29.475321 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd780-846a-4da1-8662-94db78f7f41b None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:29.475691 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd780-846a-4da1-8662-94db78f7f41b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:29.475908 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd780-846a-4da1-8662-94db78f7f41b None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:29.477466 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 674/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:29.483074 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06b55-5c9e-4af8-8fad-0e859c272073 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:29.483547 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06b55-5c9e-4af8-8fad-0e859c272073 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:29.483683 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a06b55-5c9e-4af8-8fad-0e859c272073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:29.503901 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-24a06b55-5c9e-4af8-8fad-0e859c272073 None None] MFA Rules not processed for user `57b9a2d97eff4d8aad78c189df251350`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:29.505154 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-24a06b55-5c9e-4af8-8fad-0e859c272073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:29.514846 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 679/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 19:48:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:48:30.784834 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007f04-5727-4564-9d12-02e5dea84137 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:30.785411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007f04-5727-4564-9d12-02e5dea84137 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:30.785411 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13007f04-5727-4564-9d12-02e5dea84137 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:30.786911 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 675/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 19:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:30.792234 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0597e6-2741-4756-b09e-f999c5adbe8a None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:30.793313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0597e6-2741-4756-b09e-f999c5adbe8a None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:30.793313 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0597e6-2741-4756-b09e-f999c5adbe8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:30.813359 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ea0597e6-2741-4756-b09e-f999c5adbe8a None None] MFA Rules not processed for user `57b9a2d97eff4d8aad78c189df251350`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:30.814627 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ea0597e6-2741-4756-b09e-f999c5adbe8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:30.823286 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 680/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 19:48:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:34.921594 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da08533-8ae4-4aa4-af5a-53f4f2bd1a33 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.922174 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da08533-8ae4-4aa4-af5a-53f4f2bd1a33 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.922440 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da08533-8ae4-4aa4-af5a-53f4f2bd1a33 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.924147 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 676/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:34.930066 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed158-9121-4af8-8a77-2924d9eb062d None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.930630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed158-9121-4af8-8a77-2924d9eb062d None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.930630 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ed158-9121-4af8-8a77-2924d9eb062d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.949389 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-567ed158-9121-4af8-8a77-2924d9eb062d None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:34.951395 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-567ed158-9121-4af8-8a77-2924d9eb062d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:34.959613 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 681/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:35.018231 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4b2eb-15b3-4062-b028-906ca952e851 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:35.018522 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4b2eb-15b3-4062-b028-906ca952e851 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:35.018795 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4b2eb-15b3-4062-b028-906ca952e851 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:35.020164 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 677/1358] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 19:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:35.032817 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5c31-2945-45a6-9da8-6b3e3bc182db None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:35.033693 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5c31-2945-45a6-9da8-6b3e3bc182db None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:35.035028 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d5c31-2945-45a6-9da8-6b3e3bc182db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:35.094004 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-416d5c31-2945-45a6-9da8-6b3e3bc182db None None] MFA Rules not processed for user `6196d18cd1b44984a2b1c0602745314f`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:35.101940 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-416d5c31-2945-45a6-9da8-6b3e3bc182db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:35.165564 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=100 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:35.206485 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 682/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 19:48:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:35.214540 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4b2eb-15b3-4062-b028-906ca952e851 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:35.230581 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4b2eb-15b3-4062-b028-906ca952e851 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:35.237372 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96569324-7154-4791-a396-99c748588bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6196d18cd1b44984a2b1c0602745314f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:35.238340 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569324-7154-4791-a396-99c748588bd9 service nova] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:35.238622 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569324-7154-4791-a396-99c748588bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:35.238895 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96569324-7154-4791-a396-99c748588bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:35.248773 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569324-7154-4791-a396-99c748588bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:35.250595 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96569324-7154-4791-a396-99c748588bd9 service nova] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:35.261176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 678/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:37.793906 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5eff8e-d467-4f59-a382-0514a34961a3 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.794610 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5eff8e-d467-4f59-a382-0514a34961a3 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.794610 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5eff8e-d467-4f59-a382-0514a34961a3 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.796184 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 683/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:37.801675 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e02f8-76ef-4edd-bb19-a1de30f4ea42 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.802123 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e02f8-76ef-4edd-bb19-a1de30f4ea42 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.802245 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e02f8-76ef-4edd-bb19-a1de30f4ea42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.821489 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-3a8e02f8-76ef-4edd-bb19-a1de30f4ea42 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:37.822701 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-3a8e02f8-76ef-4edd-bb19-a1de30f4ea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:37.834810 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 679/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:48:37.879712 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5eff8e-d467-4f59-a382-0514a34961a3 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:37.896347 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5eff8e-d467-4f59-a382-0514a34961a3 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:37.902086 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6196d18cd1b44984a2b1c0602745314f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:37.902734 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.903266 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.903601 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.913616 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:37.915294 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed421-1f32-4c17-90cd-b3ba20eb3c97 service nova] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:37.926959 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 684/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:40.618981 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456520aa-18cd-43f8-b5fe-741378729049 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.619492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456520aa-18cd-43f8-b5fe-741378729049 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.619492 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456520aa-18cd-43f8-b5fe-741378729049 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.620847 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 680/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:40.627399 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a757-d1b7-4118-bf51-f401801a64e4 None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.627970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a757-d1b7-4118-bf51-f401801a64e4 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.627970 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047a757-d1b7-4118-bf51-f401801a64e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.647967 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-c047a757-d1b7-4118-bf51-f401801a64e4 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:40.649216 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-c047a757-d1b7-4118-bf51-f401801a64e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:40.667125 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 685/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:40.715143 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd03eb-09f1-49af-a382-4ff450f3bce4 None None] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.715432 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd03eb-09f1-49af-a382-4ff450f3bce4 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.715984 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd03eb-09f1-49af-a382-4ff450f3bce4 None None] PATH_INFO: `/` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.718592 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 681/1366] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 19:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:40.724392 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bc22-6406-463e-aba5-4e5bd217d6bc None None] REQUEST_METHOD: `POST` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.724792 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bc22-6406-463e-aba5-4e5bd217d6bc None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.725148 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34bc22-6406-463e-aba5-4e5bd217d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.745357 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.auth.core [None req-ac34bc22-6406-463e-aba5-4e5bd217d6bc None None] MFA Rules not processed for user `6196d18cd1b44984a2b1c0602745314f`. Rule list: `[]` (Enabled: `True`). {{(pid=86703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:40.746667 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.fernet_utils [None req-ac34bc22-6406-463e-aba5-4e5bd217d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:40.757210 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 686/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 19:48:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:40.765541 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd03eb-09f1-49af-a382-4ff450f3bce4 None None] Authenticating user token {{(pid=86704) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:40.781234 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd03eb-09f1-49af-a382-4ff450f3bce4 None None] Validating token access rules against request {{(pid=86704) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:40.787534 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6196d18cd1b44984a2b1c0602745314f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86704) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:40.788213 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] REQUEST_METHOD: `GET` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.788525 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.788814 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.798902 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:40.800567 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82cb7b-1dfc-4d41-a087-29b143312ca8 service nova] RBAC: Authorization granted {{(pid=86704) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:40.816463 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 682/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:41.605897 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa54b08-7a7f-462d-943f-33d126631109 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:41.606275 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa54b08-7a7f-462d-943f-33d126631109 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:41.606530 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa54b08-7a7f-462d-943f-33d126631109 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:41.608619 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 687/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:41.614602 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ba508-0231-4ed6-b87b-5def6f6b68d6 None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:41.614917 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ba508-0231-4ed6-b87b-5def6f6b68d6 None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:41.615331 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ba508-0231-4ed6-b87b-5def6f6b68d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:41.638035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-289ba508-0231-4ed6-b87b-5def6f6b68d6 None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:41.638035 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-289ba508-0231-4ed6-b87b-5def6f6b68d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:41.648582 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 683/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:43.300113 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49c1bd-68f4-4b26-9d36-ce93bcb001e0 None None] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:43.300676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49c1bd-68f4-4b26-9d36-ce93bcb001e0 None None] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:43.300676 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f49c1bd-68f4-4b26-9d36-ce93bcb001e0 None None] PATH_INFO: `/` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:43.302022 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 688/1371] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:43.307390 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e3673-86a5-4478-a166-3232d38d1b2b None None] REQUEST_METHOD: `POST` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:43.307822 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e3673-86a5-4478-a166-3232d38d1b2b None None] SCRIPT_NAME: `/identity` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:43.307942 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e3673-86a5-4478-a166-3232d38d1b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86704) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:43.326950 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.auth.core [None req-319e3673-86a5-4478-a166-3232d38d1b2b None None] MFA Rules not processed for user `8197d33213ab455786d2c8bd2697b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86704) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:43.328176 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG keystone.common.fernet_utils [None req-319e3673-86a5-4478-a166-3232d38d1b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86704) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:43.336750 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: [pid: 86704|app: 0|req: 684/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:48:43.382220 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49c1bd-68f4-4b26-9d36-ce93bcb001e0 None None] Authenticating user token {{(pid=86703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:43.397126 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f49c1bd-68f4-4b26-9d36-ce93bcb001e0 None None] Validating token access rules against request {{(pid=86703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:43.402643 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6196d18cd1b44984a2b1c0602745314f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'da6a49f3e751495596447ab1d795277f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:43.403275 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] REQUEST_METHOD: `GET` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:43.403558 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] SCRIPT_NAME: `/identity` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:43.404642 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:43.413142 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:43.414772 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5e5cb-71b8-4885-9189-4bdc11bc61ac service nova] RBAC: Authorization granted {{(pid=86703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:43.431004 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: [pid: 86703|app: 0|req: 689/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 19:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:48:53.320953 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: DEBUG dbcounter [-] [86704] Writing DB stats keystone:SELECT=21 {{(pid=86704) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:53.426001 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: DEBUG dbcounter [-] [86703] Writing DB stats keystone:SELECT=31 {{(pid=86703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:05.937596 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: Thu May 19 19:49:05 2022 - graceful shutdown triggered... May 19 19:49:05.938057 dv-cinder-840678-11-iscsi devstack@keystone.service[86703]: Gracefully killing worker 1 (pid: 86703)... May 19 19:49:05.938461 dv-cinder-840678-11-iscsi devstack@keystone.service[86704]: Gracefully killing worker 2 (pid: 86704)... May 19 19:49:05.940186 dv-cinder-840678-11-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 19:49:06.938740 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: worker 1 buried after 1 seconds May 19 19:49:06.939282 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: worker 2 buried after 1 seconds May 19 19:49:06.939282 dv-cinder-840678-11-iscsi devstack@keystone.service[86696]: goodbye to uWSGI. May 19 19:49:06.941205 dv-cinder-840678-11-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 19:49:06.941746 dv-cinder-840678-11-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 19:55:11.702541 dv-cinder-840678-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 19:55:11.712379 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 19:55:11 2022] *** May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: nodename: dv-cinder-840678-11-iscsi May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: machine: x86_64 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: clock source: unix May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: pcre jit disabled May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: detected number of CPU cores: 4 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: current working directory: / May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: detected binary path: /usr/bin/uwsgi-core May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: your processes number limit is 47680 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: your memory page size is 4096 bytes May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: detected max file descriptor number: 2048 May 19 19:55:11.714739 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: lock engine: pthread robust mutexes May 19 19:55:11.715915 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: thunder lock: enabled May 19 19:55:11.715915 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 19:55:11.715915 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 19:55:11.769075 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: Python main interpreter initialized at 0x561c283669c0 May 19 19:55:11.769075 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: python threads support enabled May 19 19:55:11.769075 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: your server socket listen backlog is limited to 100 connections May 19 19:55:11.769075 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: your mercy for graceful operations on workers is 90 seconds May 19 19:55:11.769522 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: mapped 403077 bytes (393 KB) for 2 cores May 19 19:55:11.769522 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: *** Operational MODE: preforking *** May 19 19:55:11.769727 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: *** uWSGI is running in multiple interpreter mode *** May 19 19:55:11.769727 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: spawned uWSGI master process (pid: 150726) May 19 19:55:11.769949 dv-cinder-840678-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 19:55:11.773601 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: spawned uWSGI worker 1 (pid: 150737, cores: 1) May 19 19:55:11.773601 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: spawned uWSGI worker 2 (pid: 150738, cores: 1) May 19 19:55:11.773601 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 19:55:17.583202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:55:17.583202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: from cryptography.utils import int_from_bytes May 19 19:55:17.584089 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:55:17.584089 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: from cryptography.utils import int_from_bytes May 19 19:55:17.585128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:55:17.585128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: from cryptography.utils import int_from_bytes May 19 19:55:17.585638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:55:17.585638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: from cryptography.utils import int_from_bytes May 19 19:55:17.635967 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:55:17.636200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:55:17.636317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] command line args: [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:55:17.636617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:55:17.636798 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ================================================================================ {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:55:17.637054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] admin_token = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.637246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:55:17.637369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] config_dir = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.637473 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:55:17.637566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.637664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] command line args: [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:55:17.637755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] config_source = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.637844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] debug = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.637934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:55:17.638033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ================================================================================ {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:55:17.638279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] admin_token = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638607 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] config_dir = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638879 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.638982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] config_source = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] debug = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639304 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639411 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_config_append = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639501 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639655 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] insecure_debug = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_dir = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.639936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_file = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640059 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640173 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640683 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_config_append = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.640975 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_dir = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641154 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_file = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641340 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.641913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642411 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642817 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.642916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643407 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.643932 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644027 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] publish_errors = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] notification_format = cadf {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.644839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645071 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] publish_errors = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] standard_threads = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645526 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.645906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646005 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] use_journal = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] use_json = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] use_stderr = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] standard_threads = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] use_syslog = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] strict_password_check = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.646955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647053 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.647240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] use_eventlog = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.647447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] use_journal = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] use_json = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.647730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] use_stderr = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.647866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.647954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] use_syslog = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.648054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.648148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] watch_log_file = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:55:17.648239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.648333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.648482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.648577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.648910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.650601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.650731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.650822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.650923 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651104 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651636 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.651917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652123 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652732 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.652938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653304 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.653943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654114 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654556 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654659 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.654893 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655754 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.655867 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656745 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.656987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.657934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658417 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658765 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.658950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.659902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660032 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660181 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660484 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.660886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661031 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661843 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.661981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.662972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.663941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664527 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.664937 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665074 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.665751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.666583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.666583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.666583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.666959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667499 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.667914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.668056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.668184 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.668320 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.668485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.674618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.674782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.674782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675650 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.675941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.enabled = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676266 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.676970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677240 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677607 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.677895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678320 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678806 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.678949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679642 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.679918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.external = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680534 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.680957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.password = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681526 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] auth.token = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681662 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.681937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682208 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682340 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.682938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.683903 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684299 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.external = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684624 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.mapped = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.684886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.password = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] auth.token = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.685871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] catalog.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.686995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687407 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.687951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.688855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689416 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689416 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.689974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690370 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.690917 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691201 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.691883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.692905 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693184 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693469 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.693899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694862 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.694994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.695125 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.695266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.695398 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.695530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.696897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.697888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698311 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.698874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699006 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.699870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.700926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.701882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702098 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702362 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702486 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.702909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703051 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703184 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703467 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.703887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.704896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.705134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.705400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.705655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.705938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.706935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.707107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.707306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.707488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.707673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.password = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.707847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.708959 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.709133 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.709133 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.709481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.709654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.709843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.710016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.710189 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.710362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.710545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.710794 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.711070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.712032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.712291 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.712576 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.712776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.712958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.713147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.713328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.713523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.713714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.713891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.714086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.714290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.714483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.714684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.714879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.715067 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.715244 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.715444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.715638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.715839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716319 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.716910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717340 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717473 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.717897 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.718982 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719534 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719817 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.719972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.720929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721063 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721458 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.721853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] policy.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722719 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.722851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] role.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] role.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] receipt.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.723937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724794 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.724929 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.725764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726023 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726154 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.726884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] revoke.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727291 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727754 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.727895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] role.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728181 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] role.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] role.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728762 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] role.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.728895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729440 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.729995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.730956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731648 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.731934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732074 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732796 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.732936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733635 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.733914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734735 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.734875 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.735980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736256 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.736953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.737906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.connection = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.provider = fernet {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.738992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.739945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740091 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] trust.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.740974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741494 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741767 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.741899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.742908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743063 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.connection = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743635 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150737) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.743778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:55:17.744890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=150738) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:55:17.755374 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.755610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.755839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.759330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.759673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.759908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.765970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.769437 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.795262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.795491 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.797582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.797819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.850478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.851411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.903816 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.904068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.904373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.904637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972768 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.972866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973420 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973522 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.973712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.997575 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.997810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.997930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998253 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.998921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.999020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:17.999143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.011031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.011410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.026733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.026927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027284 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027377 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027905 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150737) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.027998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.028121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.028345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=150738) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:55:18.045764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.045977 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:55:18.046255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.046365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.046464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:55:18.046551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:55:18.047025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:55:18.047356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.047878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.048050 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.048310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.058868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.059152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.059410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.059591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.059694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.059877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.060151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.060472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.067823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.072750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.072956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.072956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.072956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.072956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.073379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.073995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074088 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.074760 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.075041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.075317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.101872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.103909 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.104054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.104342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.105625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.105921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.109742 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.110026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.110307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.111284 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.111571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.111859 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.117830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.118115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.119573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.119861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.122856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.123151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.123435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.123765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.124000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.124278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.124692 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.124833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.124929 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.125055 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.125350 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.125717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.125996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.126209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.126488 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.126965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.143989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.144290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.146896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.147199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.149420 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.152511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.153095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.153392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.156141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.156469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.158400 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.161483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.162036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.165660 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.166479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.166781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.167077 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.167365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.167706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.168018 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.169452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.169756 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.170048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.170628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.170915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.187702 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.188112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.188388 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.188681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.190763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.191179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.191459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.191739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.196690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.198440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.198721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.199870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.201656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.201830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.201926 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.202117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.202406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.202686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.202958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.205033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.205322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.205619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.205910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.206194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.217147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.217428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.217718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.218011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.218291 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.218569 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.218858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.220206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.220520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.220809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.221096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.221382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.221661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.221953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.239307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.241737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.242790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.245140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.246113 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.247728 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.248513 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.249434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.249724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.250010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.250142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.251824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.252116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.252419 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.257486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.259654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.260457 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.262095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.262913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.263207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.263497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.263786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.265414 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.265714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.266006 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.266297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.273119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.273488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.273875 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.275664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.275964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.276254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.282167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150737) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.282562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.282562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.282830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.283235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.283674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.283949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.284233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.284644 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.284831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=150738) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:55:18.284943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150737) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.285971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.286245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.286590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.286803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.287024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.287290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.287573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.287884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.288100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.288367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=150738) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:55:18.326766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561c283669c0 pid: 150737 (default app) May 19 19:55:18.329498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561c283669c0 pid: 150738 (default app) May 19 19:55:18.330518 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e19e5-287a-48c9-b0a1-b34c10d73d66 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:18.330695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e19e5-287a-48c9-b0a1-b34c10d73d66 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:18.330979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e19e5-287a-48c9-b0a1-b34c10d73d66 None None] PATH_INFO: `/v3` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:18.335365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu May 19 19:55:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:26.530671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:26.531243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:26.531243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:26.535605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:26.541671 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:26.542133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:26.542236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:26.621914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: INFO dbcounter [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] Registered counter for database keystone May 19 19:55:26.644938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG oslo_db.sqlalchemy.engines [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150737) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:55:26.651528 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writer thread running {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:55:26.841917 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG passlib.handlers.bcrypt [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150737) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:55:26.842259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG passlib.handlers.bcrypt [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150737) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:55:26.900050 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:26.901707 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:27.041093 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 19:55:27.063881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:27.134060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: INFO dbcounter [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] Registered counter for database keystone May 19 19:55:27.154907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG oslo_db.sqlalchemy.engines [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=150738) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:55:27.161456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writer thread running {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:55:27.286471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c0cc-1503-4c5a-94b2-89525a000995 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:27.291766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0700fd9a2e34b6cbec11fb0d6875eef', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:27.292452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:27.292746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:27.293035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:27.302997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:27.360540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: WARNING keystone.common.rbac_enforcer.enforcer [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:55:27.360892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a20b8b-f98d-4fd8-92e9-f17e7fefcdce admin admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:27.371536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 19:55:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:55:27.377554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:27.392730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39601a30-6df1-4d67-9e10-9e88ebd0f0d1 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:27.397321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0700fd9a2e34b6cbec11fb0d6875eef', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:27.398649 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:27.399391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:27.400293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] PATH_INFO: `/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:27.402029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] RBAC: Authorizing `identity:get_project(project_id=a0700fd9a2e34b6cbec11fb0d6875eef)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:27.573284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:55:27.573977 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb6827-b8e6-4d45-a475-66f75b547231 admin admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:27.578391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu May 19 19:55:27 2022] GET /identity/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:29.814072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3c3f-79c3-4926-ab47-8b45aede9640 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:29.814608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3c3f-79c3-4926-ab47-8b45aede9640 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:29.814608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f3c3f-79c3-4926-ab47-8b45aede9640 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:29.818721 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:29.825044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:29.825505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:29.825609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:29.853752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:29.855431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:29.863848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:55:29.886628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3c3f-79c3-4926-ab47-8b45aede9640 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:29.900802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f3c3f-79c3-4926-ab47-8b45aede9640 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:29.905492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0700fd9a2e34b6cbec11fb0d6875eef', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:29.906173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:29.906456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:29.906738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:29.915450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:29.917293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53985eac-e56f-44a9-8852-7619c2399d8e admin admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:29.926489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 19:55:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:55:29.933094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:29.942205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faf796-b991-4c0c-9b03-89c1e43d18a2 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:29.947478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a0700fd9a2e34b6cbec11fb0d6875eef', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:29.948349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:29.948660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:29.948936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:29.950496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:29.952377 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4754b-c337-4b57-ab61-d323dad31c44 admin admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:29.954097 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu May 19 19:55:29 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:32.307910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d8a99-8890-43c3-8d24-f1b474e7ba45 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.308537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d8a99-8890-43c3-8d24-f1b474e7ba45 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.308537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d8a99-8890-43c3-8d24-f1b474e7ba45 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.309860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:32.315233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.315656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.315804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.323123 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93992-ddce-4f23-8983-391d0ea9fa63 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.323474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93992-ddce-4f23-8983-391d0ea9fa63 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.323764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93992-ddce-4f23-8983-391d0ea9fa63 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.325156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:32.330414 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.330706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.330988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.333785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:32.334994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:32.373034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG passlib.handlers.bcrypt [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=150738) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:55:32.373455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG passlib.handlers.bcrypt [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=150738) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:55:32.420072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:32.422021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:32.423668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:32.447893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:32.473384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:32.530237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:32.558691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:32.562192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003ef13-e4fd-43c0-9a71-fbb6c28edbc5 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:32.598658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:32.599485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.599774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.600058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.624368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:32.637237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:32.638460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:32.639481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:32.646886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:32.739767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:32.759778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2673425-a105-4df9-a487-e95533bd8fda None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:32.784064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2e8ed-2d99-468e-85f5-519a4bcbb47c None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:32.786528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:32.797830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:32.798514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.798802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.799082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.799845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:32.800157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:32.801881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:32.814460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:32.820085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:32.828388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:32.829343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.829635 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.829918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.903731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:32.905317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:32.943158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d700b61-c023-49d7-a4fa-1a76bb567613 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:32.987385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:32.988277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:32.988617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:32.988902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:32.990195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:32.991790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:32.993979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:32 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:32.999584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:33.016132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:33.021972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 8/17] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:55:33.027949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:33.031842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:33.106403 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702baa0-1c94-4a62-9568-5f4f6b3b5793 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:33.141120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb102f-7786-4d6d-93d1-fb81edcc6aaa None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:33.150773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:33.151603 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.151892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.152180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.159008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:33.160622 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:33.162304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:33.169241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:33.183115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:33.183957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.184227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.184551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.206809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:33.208291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:55:33.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:33.230355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:33.231667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:55:33.231667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:55:33.231667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:55:33.231667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:33.231667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:33.236751 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:33.263254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:33.283365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ba229-326d-4689-afbc-017f94dd44ae None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:33.299511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:33.322746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:33.323414 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.323701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.323982 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.324866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.domain.id=default, user.id=af21adf4b88843c7988a1c2761d5763b) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:33.325178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:33.326643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df056e3b-7264-4416-a7fc-c652553d65af None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:33.333666 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 19:55:33 2022] GET /identity/v3/role_assignments?user.id=af21adf4b88843c7988a1c2761d5763b&scope.domain.id=default&role.id=771c065274f04f49bcd4641d79a0e490 => generated 233 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:33.351355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f432b6-6433-485a-a0ad-adc77a23291c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:33.390277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:33.391040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:33.391269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:33.391547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:33.392319 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:33.392672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:33.394337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7011d27-385c-4ae5-8c65-797594372b91 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:33.414933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:36.130161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8d5f-966f-437f-9463-2a94b6f2657e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.130691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8d5f-966f-437f-9463-2a94b6f2657e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.130802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f8d5f-966f-437f-9463-2a94b6f2657e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.132310 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:36.138741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.139286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.139286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.165699 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:36.167067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:36.235575 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e827-aa5e-402d-b2b8-bc1bf3c54f4c None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.235948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e827-aa5e-402d-b2b8-bc1bf3c54f4c None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.236140 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12e827-aa5e-402d-b2b8-bc1bf3c54f4c None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.237721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:36.245918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.246211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.246499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.274836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:36.276048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:36.294769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 11/24] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:36.339657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:36.418385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 14/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:36.424515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:36.449394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:36.558018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529bd7f-b5ab-4ed3-88fa-3165b8296732 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:36.631185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:36.632019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.632307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.632617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.644809 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:36.658180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:36.659688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-cec55cf4-78cf-4a71-b505-8c463b93fef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:55:36.702562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:36.703561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:36.704554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:55:36.704554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:55:36.704554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:55:36.704554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:36.704554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 12/26] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:36.705173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231084-56a4-45f6-84f9-b003fa2b21c2 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.705698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231084-56a4-45f6-84f9-b003fa2b21c2 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.705877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231084-56a4-45f6-84f9-b003fa2b21c2 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.707772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 13/27] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:36.710312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120b54b-a77e-472f-b099-d60d99dd96ce None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:36.715565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bdf41-7e9e-455f-9935-92d64ba1c672 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.715855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bdf41-7e9e-455f-9935-92d64ba1c672 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.716141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bdf41-7e9e-455f-9935-92d64ba1c672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.747942 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-426bdf41-7e9e-455f-9935-92d64ba1c672 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:36.749181 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-426bdf41-7e9e-455f-9935-92d64ba1c672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:36.787748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:36.788597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.788887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.789844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.813798 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:36.825163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:36.827845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:55:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:36.830348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:36.873438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 14/29] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:36.875204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b437-1ebf-4088-b9dc-9450e7bb3a47 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.875755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b437-1ebf-4088-b9dc-9450e7bb3a47 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.876189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b437-1ebf-4088-b9dc-9450e7bb3a47 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.879333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 15/30] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:36.880924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b437-1ebf-4088-b9dc-9450e7bb3a47 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:36.917388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:36.949730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd759d9e-e7e8-4687-9ba0-7ce4bcd4ddb4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:36.986884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:36.987883 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:36.988605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:36.989071 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:36.990551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:36.991059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:36.993296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e73bc-5349-4494-ad90-091e55cfbac5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:36.996121 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.015037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b437-1ebf-4088-b9dc-9450e7bb3a47 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.018782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 16/31] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:37.020308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.020788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.021244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.044357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:37.046408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:37.061293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.062314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.062755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.063186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.064313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:37.064949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.066544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.083292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:37.084644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.135708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 17/33] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:37.136318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.218694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.232577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f1325-6e1e-4259-bbca-ed74d25a58c7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.276589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.276799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a96f46-782f-4a27-8189-b4c14f68c4a5 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.277677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.278126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.278615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.279640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.281267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.303991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.319022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.320099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.321323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.321825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.322765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.324784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.437156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 17/34] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:37.438518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.504071 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 18/35] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/projects => generated 301 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:37.509859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.579626 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a6ca9-920f-4096-abb1-21b9a2dfb094 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.643423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.644810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.645325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.645853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.649671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.653170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.659119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 18/36] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:37 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:37.671562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:37.747307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.783262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179e2d-b8ad-4fe1-9636-5a474386954d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.832188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.833280 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.833823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.834324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.835341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.837268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c356-b291-42db-88f3-b95bf5a6468f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.856931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 19/37] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/roles => generated 225 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:37.858718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c74501-0dbd-4969-9411-f61390b5fb9e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.859249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c74501-0dbd-4969-9411-f61390b5fb9e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.859748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c74501-0dbd-4969-9411-f61390b5fb9e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.861342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 20/38] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:37.874144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c86e-fe49-4e24-846f-a6081e60ab24 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.874144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c86e-fe49-4e24-846f-a6081e60ab24 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.874144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab68c86e-fe49-4e24-846f-a6081e60ab24 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.874144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:37.874596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ae21e-18eb-4033-93d3-2eada279ac3d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:37.879755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.880098 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.880334 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.921225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:37.927651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:37.947964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:37.949787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.950477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.951781 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.955658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:37.957853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac3ad-be21-4c1f-80fa-349ef00c0ac3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:37.961138 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 19/40] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:37.963749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:37.964511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:37.964998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:37.974171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:37.993710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:37.995334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:38.032695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 22/41] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:38.032695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:38.078689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 20/42] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:38.079865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:38.105460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:38.171486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818adcd-f0c2-429a-95f1-f313b89eecde None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:38.202555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b367a-f2e2-4bae-a2fb-a0b35a25bea2 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:38.213948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:38.214835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:38.215122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:38.215411 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] PATH_INFO: `/v3/domains/default/users/af21adf4b88843c7988a1c2761d5763b/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:38.217468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=af21adf4b88843c7988a1c2761d5763b, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:38.218948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:38.233585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 23/43] 10.222.0.13 () {58 vars in 1361 bytes} [Thu May 19 19:55:38 2022] PUT /identity/v3/domains/default/users/af21adf4b88843c7988a1c2761d5763b/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:38.234390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:38.250996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:38.251939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:38.252502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:38.252993 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:38.255325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:38.258774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:38.262219 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 21/44] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:38.268352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:38.277254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:38.374519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379472b3-13ad-4924-b3a7-417a7fe1d4a7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:38.377068 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:38.395442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ded935-ee51-4909-9a45-d507f10e2580 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:38.417835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:38.418537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:38.418826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:38.419107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:38.419888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:38.421367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423c67e-d79e-498e-a86b-0aeef1c6ed2b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:38.436765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 24/45] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:55:38 2022] POST /identity/v3/roles => generated 223 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:38.440446 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:38.441151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:38.441450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:38.441723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:38.442471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:38.445487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8499e61-42a8-4af4-b91e-2693eb2818f3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:38.480562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 22/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:55:38 2022] POST /identity/v3/projects => generated 312 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:55:40.547870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6246-845d-48b0-904e-3af47e97d7a1 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.548452 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6246-845d-48b0-904e-3af47e97d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.548581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e6246-845d-48b0-904e-3af47e97d7a1 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.549872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 25/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:40.555449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f19c13-60a2-4220-8609-4f7f7e369ef5 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.555875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f19c13-60a2-4220-8609-4f7f7e369ef5 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.556023 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f19c13-60a2-4220-8609-4f7f7e369ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.572824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-f9f19c13-60a2-4220-8609-4f7f7e369ef5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:40.574034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-f9f19c13-60a2-4220-8609-4f7f7e369ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:40.650585 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 23/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:40.674218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e6246-845d-48b0-904e-3af47e97d7a1 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:40.768001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:40.784593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e6246-845d-48b0-904e-3af47e97d7a1 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:40.794887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fcceb-70f9-48b7-8a56-63e2f41484d8 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.795307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fcceb-70f9-48b7-8a56-63e2f41484d8 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.795594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fcceb-70f9-48b7-8a56-63e2f41484d8 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.797284 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 24/49] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:40.802648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.802942 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.803230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.820430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:40.821628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:40.821964 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:40.822840 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:40.823136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:40.823414 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:40.834025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:40.842185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:40.843363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:40.844288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 26/50] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:40.849585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:40.886282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:40.895955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 25/51] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:40.921266 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:40.968832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd846a6-6269-448f-9e2f-71cb8d736a82 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.021543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.021961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.022262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.022567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.024013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:41.024509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.025943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.036840 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 27/52] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:41.041707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559490a-8339-4e6f-9283-b07d1a2001d0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.042614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:41.056028 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:41.074988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.075800 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.076094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.076382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.077615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.079090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.080786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:41.087189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:41.153810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548a783-a42b-4904-a521-f906806cca7d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.163930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:41.189987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.190777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.191065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.191353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.192609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.194101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.195854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 28/54] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:41 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:41.201309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:41.206441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c3275-08b1-4a3d-b336-c8da8e7226b1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.243982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.244701 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.245000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.245286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.246007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.247484 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.280902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 27/55] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 19:55:41 2022] POST /identity/v3/projects => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:55:41.293616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:41.325121 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bebbc13-7612-42c7-a810-960c3a45ec10 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.364117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.365008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.365303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.365591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.366975 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.369060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b75b3-1b2e-4f8d-b99f-b151fd853df1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.370803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 29/56] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:41.376895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:41.496894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46cc73-41ed-46bd-b21a-29726da45845 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:41.524574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:41.545162 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:41.545754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:41.546047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:41.546331 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:41.547441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.domain.id=default, user.id=af21adf4b88843c7988a1c2761d5763b) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:41.547759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:41.549248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67fcb2-b014-4401-b1bf-dd9a2dfde0ed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:41.558710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 28/57] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 19:55:41 2022] GET /identity/v3/role_assignments?user.id=af21adf4b88843c7988a1c2761d5763b&scope.domain.id=default&role.id=771c065274f04f49bcd4641d79a0e490 => generated 530 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:43.574223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fde4e-2781-46dc-b6f9-fcdc3f69c54c None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:43.574820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fde4e-2781-46dc-b6f9-fcdc3f69c54c None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:43.574820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fde4e-2781-46dc-b6f9-fcdc3f69c54c None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:43.576180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:43.581476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:43.581982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:43.582080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:43.599243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:43.600513 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:43.677402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:43.702511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fde4e-2781-46dc-b6f9-fcdc3f69c54c None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:43.825965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fde4e-2781-46dc-b6f9-fcdc3f69c54c None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:43.866279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:43.867111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:43.867408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:43.867698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:43.868959 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:43.870462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9a559-4c7e-4324-ba93-d322d4604ab1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:43.872128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:43.877840 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:43.989240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612f119-3516-4274-8a1f-09e8c5acf4a0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:43.999908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:44.034230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:44.034907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:44.035195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:44.035475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:44.036191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:44.037681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1982-1cfa-4587-aaf6-83e12846c4dd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:44.062607 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:55:43 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:47.698343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f502f-ec02-47ab-97a0-c80ed1b3669d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.698894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f502f-ec02-47ab-97a0-c80ed1b3669d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.698894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f502f-ec02-47ab-97a0-c80ed1b3669d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.700470 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:47.714734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.715269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.715269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.741489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:47.742957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:47.784060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f2f0f-169c-4c0a-8407-2624d8b441db None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.784246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f2f0f-169c-4c0a-8407-2624d8b441db None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.784529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f2f0f-169c-4c0a-8407-2624d8b441db None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.785958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 33/63] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:47.800061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a520e-f50b-4f05-b051-d0e62f8d7c51 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.800788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a520e-f50b-4f05-b051-d0e62f8d7c51 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.801316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a520e-f50b-4f05-b051-d0e62f8d7c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.838098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 31/64] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:47.849458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-d48a520e-f50b-4f05-b051-d0e62f8d7c51 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:47.850687 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-d48a520e-f50b-4f05-b051-d0e62f8d7c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:47.862917 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:47.865500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:47.933116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 34/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:47.934469 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de563435-8e9a-4169-8c84-4eac5a1de087 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.934756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de563435-8e9a-4169-8c84-4eac5a1de087 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.935038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de563435-8e9a-4169-8c84-4eac5a1de087 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.936318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 35/66] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:47.941608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:47.942084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:47.942539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:47.961343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:47.962919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.001848 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.009711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24832fce-cae1-408d-8603-0cd68e89afd6 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.043141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 36/67] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:48.045299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.059043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.060200 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.060820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.061302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.075480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f6ade76e-b77d-417c-b1a8-47427cc76305 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:48.085506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:48.087279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:48.088540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 32/68] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:55:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:48.089567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459ddbc-3b1c-4bfe-b9f4-cb870cc9de65 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.089567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459ddbc-3b1c-4bfe-b9f4-cb870cc9de65 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.089567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459ddbc-3b1c-4bfe-b9f4-cb870cc9de65 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.095191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 33/69] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:48.095191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459ddbc-3b1c-4bfe-b9f4-cb870cc9de65 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.132501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.171732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bc009-3dcc-423f-90cd-da08748d63ad None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.218778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.219591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.219883 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.220170 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.231240 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:48.232218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459ddbc-3b1c-4bfe-b9f4-cb870cc9de65 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:48.239407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:48.240543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:48.241474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 37/70] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:48.244846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.274914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.275736 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.276033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.276322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.296752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c7b8de0c-9d7e-4dbd-bd5f-936316621cc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:48.305477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:48.306480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:48.307543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 34/71] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:48.310337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e843edd-cc70-4acc-b10c-1f0e0d6f448a None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.310632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e843edd-cc70-4acc-b10c-1f0e0d6f448a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.310920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e843edd-cc70-4acc-b10c-1f0e0d6f448a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.329653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-7e843edd-cc70-4acc-b10c-1f0e0d6f448a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:48.331038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-7e843edd-cc70-4acc-b10c-1f0e0d6f448a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.348687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.358819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1087d9c-6173-4f54-a518-89b71212a6b5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.394267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.400646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.401331 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.401624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.401910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.402671 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:48.402987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.404415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.412732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 38/72] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:48.413536 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.420270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 35/73] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:48.421647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f93f97-d1d8-47b5-bcb8-075326e725e0 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.422187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f93f97-d1d8-47b5-bcb8-075326e725e0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.422476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f93f97-d1d8-47b5-bcb8-075326e725e0 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.423743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 36/74] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:48.428462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f93f97-d1d8-47b5-bcb8-075326e725e0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.532249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7109a2eb-49a3-404b-a6f2-380c4142f8f4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.545690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f93f97-d1d8-47b5-bcb8-075326e725e0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.569294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.569887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.570186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.570472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.571324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:48.571645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.573150 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.581572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 39/75] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:48.581572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.586438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.586438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.586438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.586438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.586807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:48.586807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.586807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f34cf-ce33-4bc1-af6e-a92eccb02af7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.602074 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 37/76] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:48.604213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.604839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.605153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.622558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:48.623788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:48.625834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.659078 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.704265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 38/77] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:48.705124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.724825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de5c3d-6c19-41ef-ae04-bba43b768cf6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.761952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.762851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.763142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.763427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.777598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.779095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.780826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 40/78] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:48.785492 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.811977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362517b-0347-4345-9f06-ac2e2910de22 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.848884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.849598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.849891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.850180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.857979 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.862922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:48.871474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:48.872694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 39/79] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:48.873714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.899261 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaa834-7b25-4cb4-9ff9-c810eada669a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:48.915259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:48.938167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:48.939051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:48.939348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:48.939635 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:48.940908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:48.942468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:48.944036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 41/80] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:48.944965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:48.986393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea1beb-58ec-4826-88ba-c192b2d94d92 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.022045 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.022940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.023251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.024054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.025384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.026864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.028554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 40/81] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:49.029338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.060419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49589d64-a50d-4bbd-9e72-12934d5cf37f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.097424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.097661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.098238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.098558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.098935 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.100134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.101690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.103521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 42/82] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:49.104132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.138797 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509cb5-ccf7-4c25-b4d2-21e26ede33fc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.158842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.174434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.175103 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.175396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.175745 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.214976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d1c4f-02e5-4c22-b342-9191271d479c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.246774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.248262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.258374 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.259137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.259428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.259710 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.260509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:49.260824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.262319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.270735 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 43/83] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:49.274009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.317569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.358704 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 41/84] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:55:49.360129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.384260 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6676ec-351e-4544-a11e-e5c3587b61ce None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.397670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.419808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.420498 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.420789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.421074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.467430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949e5af-ea59-425c-8182-4289d3cb689f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.494167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.495682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.503508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.504209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.504533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.504915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.542186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.579812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.581405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.618512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 44/85] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:55:49.619271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.634454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.687891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 42/86] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:55:49.689369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.726560 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004730-eb85-47d4-a5cf-a777db7f15b9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.764675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.765358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.765647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.765931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.766155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.766645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.768122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.794472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 45/87] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:55:49 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:55:49.795390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.815085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4abccb-8301-4b9e-9a24-474e817568e7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.860721 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.861641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.861931 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.862209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.863859 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.865504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.868141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 43/88] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:49.869174 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:49.884863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:49.930928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aea0a8-a6ea-41ed-bcd3-a50711482873 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:49.978937 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:49.979833 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:49.980123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:49.980435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:49.981686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:49.983176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:49.984954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 46/89] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:49.985745 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.009989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502fd5-0ed5-49b5-9f71-01d0269a5563 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.065637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.066547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.066842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.067128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.068497 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.070573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.072548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 44/90] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:50.073351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.076191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:50.144773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342980f-1fed-41d8-9e69-6ea76e7df47b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.197451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.198630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.198971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.200211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.201744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.203487 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 47/91] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:49 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:50.204277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.215985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cccf9-3bd4-46de-ba1f-2a258e24e2dc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.240801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:50.261297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.262498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.262893 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.263595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.352456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853249a9-b065-4a0a-ba3a-163c9bdd71b1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.353014 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.354681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.368976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:50.400341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.400965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.401256 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.401538 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.402689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:50.403047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.405545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.412953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 48/92] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:55:50 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=dd3a79631b06401fab3c7670acb9024d => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:50.414269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.525282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 45/93] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:55:50.526334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.558833 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:50.605606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2effb7c-9ca5-4258-8267-d0915c2a86e2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.652228 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.653027 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.653327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.653615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.654825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:50.655158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.656961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.667567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 49/94] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:55:50 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=afa620995009427092fc6164c9cc643b => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:50.668277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:50.761234 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75a3e6-5336-4bf9-ae2c-f8329b1fca15 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:50.764835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:50.883592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:50.884891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:50.885562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:50.885729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:50.887211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=af21adf4b88843c7988a1c2761d5763b) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:50.887761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:50.894716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:50.915720 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 46/95] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:55:50 2022] GET /identity/v3/role_assignments?user.id=af21adf4b88843c7988a1c2761d5763b&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:50.973231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4712e-1e38-441f-99a3-b1bcf368725f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:51.033678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:51.042853 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:51.045110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:51.046112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:51.046909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] PATH_INFO: `/v3/projects/2415b8fb72a74388ba4695e700a86a3e` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:51.049994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] RBAC: Authorizing `identity:get_project(project_id=2415b8fb72a74388ba4695e700a86a3e)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:51.053459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78cc21-f20e-4e81-b6e5-941c56169991 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:51.058054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 50/96] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:50 2022] GET /identity/v3/projects/2415b8fb72a74388ba4695e700a86a3e => generated 312 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:51.064980 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:51.252711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acb9e3-316e-40de-bd7e-0a65daf57127 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:51.291898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:51.292603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:51.292899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:51.293183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:51.294034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=2415b8fb72a74388ba4695e700a86a3e, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:51.294362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:51.295834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dde6f2-3c68-4752-bc55-f9fe857586b4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:51.302888 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 47/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:55:51 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=2415b8fb72a74388ba4695e700a86a3e&role.id=afa620995009427092fc6164c9cc643b => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:53.138386 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b4452-e31c-4adc-a7db-3e7b847f9127 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.139339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b4452-e31c-4adc-a7db-3e7b847f9127 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.139794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b4452-e31c-4adc-a7db-3e7b847f9127 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.141383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 51/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:53.146989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.147460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.147774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.170873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:53.172351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:53.213661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:53.234368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600e89-99b5-4d3c-8f00-00313ffaeaa3 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.235059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600e89-99b5-4d3c-8f00-00313ffaeaa3 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.235563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600e89-99b5-4d3c-8f00-00313ffaeaa3 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.245398 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 52/99] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:53 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:53.251217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.251783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.252273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.271076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 48/100] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:53.280528 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:53.282082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:53.355681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:53.387336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 53/101] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:53.431730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:53.551332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdbde11-f152-4499-8b52-96595769a491 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:53.612166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:53.613408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.613878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.614341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.615951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25392b5c-b057-42bf-936c-239d8bf779e8 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:53.619317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:53.620095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:53.621860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 49/102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:53.627217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:53.653059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:53.668006 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:53.668866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.669158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.669443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.688773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:53.708096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:53.709317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:53.710298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 54/103] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:53.713456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:53.716017 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:53.876600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db23b8-5eea-462f-8c39-e16c21c8365e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:53.918091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:53.918819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.919112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.919391 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.920120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:53.921764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413a77a-df53-415b-951f-b34b627fd57a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:53.954842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 50/104] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:55:53 2022] POST /identity/v3/projects => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:55:53.956991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeada0d-54d3-4a87-8591-b096274c5157 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.957369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeada0d-54d3-4a87-8591-b096274c5157 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.957672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeada0d-54d3-4a87-8591-b096274c5157 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.959695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 51/105] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:53.967144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:53.968104 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:53.968427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:53.972724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ce0d7-dbf3-4309-ad93-bb0480fb7dba None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:53.990333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:53.992530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:54.066330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.067115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.067417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.067731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.073562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:54.073931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:54.075372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:54.090028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 52/106] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:55:54.094831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 55/107] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:54.100483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.118060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.143524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.231741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44751c20-3541-4cc0-82a3-472d01098d8a None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.248055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.285507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.286417 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.286705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.286982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] PATH_INFO: `/v3/users/7e6273b9f6c24071abf2853f89ab26dc` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.326388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] RBAC: Authorizing `identity:get_user(user_id=7e6273b9f6c24071abf2853f89ab26dc)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:54.328368 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd433a7b-2a37-468d-b0cb-4a4ee8771e1f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:54.333606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 53/108] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/users/7e6273b9f6c24071abf2853f89ab26dc => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:54.335163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cadfb-5b9d-44fd-b713-2d97dcb7fcaa None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.335504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cadfb-5b9d-44fd-b713-2d97dcb7fcaa None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.336522 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44cadfb-5b9d-44fd-b713-2d97dcb7fcaa None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.337879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 54/109] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:54.338659 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cadfb-5b9d-44fd-b713-2d97dcb7fcaa None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.365222 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab971a-b046-4008-81a5-abebd2698532 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.423352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.424181 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.424499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.424784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.440081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:54.443257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-19a0dd01-3162-4761-bb7b-931ed976b7af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:54.452287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:54.453402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:55:54.454294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 56/110] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:55:54.455750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.456146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.456562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.491064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:54.493701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:54.509145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44cadfb-5b9d-44fd-b713-2d97dcb7fcaa None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.579174 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.579870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.580207 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.580572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.663508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:54.665096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d1bb2-4051-423f-b201-9cd7c2313dc0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:54.680531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 57/111] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:54.681504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.737750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.746622 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:54.776424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 55/112] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:55:54.777863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36967-dc25-4482-8e17-d1d64e6c0516 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.778227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36967-dc25-4482-8e17-d1d64e6c0516 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.778612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36967-dc25-4482-8e17-d1d64e6c0516 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.783908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 56/113] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:54.783908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b36967-dc25-4482-8e17-d1d64e6c0516 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.818957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dde99-7712-4afb-afc6-59d84a0dd8cc None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.859120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.860242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.860783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.861304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.862295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:54.862826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:54.864594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:54.874423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:54.875751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:54.899062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b36967-dc25-4482-8e17-d1d64e6c0516 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:54.940266 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:54.941163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.941453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.941732 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.955298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-45d5d00a-1f6d-4ea3-b1ce-2f42a35f359d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:54.964089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:54.965307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:54.970953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:54.971786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:54.971786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:54.988691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:54.990034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:55.006801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0de9e-0d67-4ebd-be4a-726d7888e68a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.010675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.016854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.063979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.065691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.066268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.066560 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.068003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.069615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.069738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 58/116] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:55.071297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.072224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 59/117] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:54 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.073066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.199616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6380fda8-ff8e-47c2-8710-d1fe8481ede2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.207079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680abfd7-31d6-4d75-ba86-5ff0d615468e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.238215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.238900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.239218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.239474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.240230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:55.240578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.242085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.245536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.246432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.246727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.247010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.248231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.249739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.257171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 60/118] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.257339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.257475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.258067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.291097 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.298367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.373210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ac223-fc62-440e-aef3-7e130bd9f747 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.380804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e111f-a0d3-49a2-81d7-47c8f1d98697 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.414985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.415710 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.416008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.416290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.417238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=7e6273b9f6c24071abf2853f89ab26dc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:55.417566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.419083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3c3f3-2a2d-4fd5-8d82-6af7bba02d15 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.420363 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.421675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.422628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.422975 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.426482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 61/120] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/role_assignments?user.id=7e6273b9f6c24071abf2853f89ab26dc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.428181 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4eb44-9340-4531-b64e-aad468db1e11 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.428503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4eb44-9340-4531-b64e-aad468db1e11 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.428779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4eb44-9340-4531-b64e-aad468db1e11 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.435123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 62/121] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:55.435123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4eb44-9340-4531-b64e-aad468db1e11 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.442492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:55.451161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:55.452280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:55.453604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 60/122] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:55.458245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.559788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.576639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73534d65-f55f-45bd-ac46-f07c2e0c8056 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.588160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.596170 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4eb44-9340-4531-b64e-aad468db1e11 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.616200 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.617835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.618220 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.618599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.620008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.621654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cde5c1-ab6e-43c8-b863-e1610582ba97 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.623551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 61/123] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.630282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.630674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.643613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.644909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.645404 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.645900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.647361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.649602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.649913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:55.651151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:55.651904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.652894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.736623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 62/125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:55.738010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.811004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848aa6-6427-48ad-8d38-50448575ceee None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.842081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.861764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.862171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd3e59-3709-43e1-9e73-f47605f9b5c8 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:55.862949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.863406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.864114 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.866042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:55.866545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.868376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.881081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 64/126] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:55.882292 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:55.909846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:55.910746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:55.911033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:55.911308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/4e2602f304894d35a33730dfcd6cc0cc/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:55.914927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, user_id=4e2602f304894d35a33730dfcd6cc0cc, role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:55.916535 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:55.928234 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:55.941895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 63/127] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:55:55 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/4e2602f304894d35a33730dfcd6cc0cc/roles/dd3a79631b06401fab3c7670acb9024d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:55.941895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.077021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da720a80-fa3d-4bf1-9101-d630b6989ac4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.091681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbdc03-e5b7-4c92-b1eb-0d6a357725dd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.141275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.142193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.142482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.142766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.147995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:56.153800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:56.158913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 65/128] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:56.159780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.188176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.189101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.189404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.189676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.208041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:56.217267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:56.218360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:55:56.219302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:56.222061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 64/129] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:56.223210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.229342 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:56.330902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:56.339786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bda500-cdac-4dd3-bd8a-96d3f94d2e3e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.353804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e012-ff7a-43f6-85b2-1b372fba7772 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.400733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.403338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.404736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.405458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.408607 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:56.410691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:56.415645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 66/130] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:56 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:56.421781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.427153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.430480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.431033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.431546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/4e2602f304894d35a33730dfcd6cc0cc/roles/afa620995009427092fc6164c9cc643b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.437453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, user_id=4e2602f304894d35a33730dfcd6cc0cc, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:56.439203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:56.464755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 65/131] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:55:56 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/4e2602f304894d35a33730dfcd6cc0cc/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:56.466305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.653767 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:56.690654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a3b60-5f7d-478d-930d-01fb76451580 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.727902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0330-a4f2-426a-b86c-e123cd13d54e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:56.766395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.767074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.767364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.767646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.768680 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:56.771017 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:56.772637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:56.796920 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 67/132] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:56.805826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.821828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:56.824428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:56.824983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:56.825470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:56.829653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:56.832710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:56.838426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 66/133] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:56 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:56.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:56.888960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.026517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94800-1ca0-47c2-b7fc-96bc006bc3fb None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.080541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9af9e0-0e48-4c28-ae36-825bf5841d5c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.116839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.127742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.129143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.129850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.130875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/af21adf4b88843c7988a1c2761d5763b/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.134754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, user_id=af21adf4b88843c7988a1c2761d5763b, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.137984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9102bf8-49c2-4d2e-8bd6-dba245b8313d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.138864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.140012 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.141106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.141673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.145427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.147327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c0d06-fec1-4d70-8281-de4eb88f4b38 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.151754 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 68/134] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:56 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:57.155380 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9f28-c6d8-4ef7-b118-a4c17ed745fa None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.156155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9f28-c6d8-4ef7-b118-a4c17ed745fa None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.156698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a9f28-c6d8-4ef7-b118-a4c17ed745fa None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.158378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 69/135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:57.166470 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a9f28-c6d8-4ef7-b118-a4c17ed745fa None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:57.173099 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 67/136] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:55:56 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/af21adf4b88843c7988a1c2761d5763b/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:57.174769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.177004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.178421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.209759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:57.212198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:57.348541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 68/137] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:57.390842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.408462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:57.411359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a9f28-c6d8-4ef7-b118-a4c17ed745fa None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.497408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.498451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.498605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.498861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] PATH_INFO: `/v3/projects/2415b8fb72a74388ba4695e700a86a3e` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.506198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] RBAC: Authorizing `identity:get_project(project_id=2415b8fb72a74388ba4695e700a86a3e)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.508108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.511071 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 70/138] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:57 2022] GET /identity/v3/projects/2415b8fb72a74388ba4695e700a86a3e => generated 312 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:57.524886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:57.574514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052eae1-3614-493b-82f8-cc7931e5dcdc None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.604978 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:57.623510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.624666 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.625151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.626198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.628133 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.630264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.634750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 69/139] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:55:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:55:57.640897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:57.781110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1a16-6cc8-4a29-a767-5f938d84f5e6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.832101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.832710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.833229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.834249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.836043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768abc0-df1f-4691-9cb0-9c9e78829a02 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.867931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b4e5a-36ca-4521-bc10-1b28f055ff54 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:57.880532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 70/140] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:55:57 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:55:57.935372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:57.936323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:57.936642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:57.936925 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] PATH_INFO: `/v3/projects/2415b8fb72a74388ba4695e700a86a3e/users/4e2602f304894d35a33730dfcd6cc0cc/roles/afa620995009427092fc6164c9cc643b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:57.942077 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] RBAC: Authorizing `identity:create_grant(project_id=2415b8fb72a74388ba4695e700a86a3e, user_id=4e2602f304894d35a33730dfcd6cc0cc, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:57.943595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ac3c9-53f7-4ea3-a318-b76028a4f0ba None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:57.969865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 71/141] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:55:57 2022] PUT /identity/v3/projects/2415b8fb72a74388ba4695e700a86a3e/users/4e2602f304894d35a33730dfcd6cc0cc/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:58.609470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223884d-f7d1-42e9-9afb-1851d6b4072b None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:58.610034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223884d-f7d1-42e9-9afb-1851d6b4072b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:58.610140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223884d-f7d1-42e9-9afb-1851d6b4072b None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:58.611581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 71/142] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:58.617990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:58.618506 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:58.618598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:58.644499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:58.645718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:58.669158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:58.738293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 72/143] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:55:58.772039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223884d-f7d1-42e9-9afb-1851d6b4072b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:58.830435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:58.898152 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223884d-f7d1-42e9-9afb-1851d6b4072b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:58.944657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:58.946307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:58.946608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:58.946894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:58.958459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:55:58.974409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:55:58.975580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:55:58.978366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 72/144] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:55:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:55:58.982444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:59.117936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eda62-c7df-4165-8de8-8067e370aaee None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:59.164762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:59.165456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.165740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.166020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.166784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:55:59.167092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:59.168543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:59.178916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 73/145] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:55:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:59.185081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:59.359785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b7b25-e16f-4d45-8c90-299b0739c7b7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:59.374919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:59.427958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:59.428885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.429182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.429541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] PATH_INFO: `/v3/users/7e6273b9f6c24071abf2853f89ab26dc` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.433174 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6273b9f6c24071abf2853f89ab26dc)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:59.434759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:59.436956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 73/146] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:55:59 2022] GET /identity/v3/users/7e6273b9f6c24071abf2853f89ab26dc => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:59.445859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:59.468376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:55:59.586865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eda80ef-16fc-4916-a9e1-c25e1d3eb943 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:59.631908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:59.633472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.633993 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.635279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:59.636790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bc99c-d43f-4322-b892-283f508ebc68 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:59.638611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 74/147] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:55:59 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:55:59.644745 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:55:59.878596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599b569-ac16-4fdb-b212-fcc3cdcae090 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:55:59.891547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed08f0-e6b8-4eb3-958f-a7f4b57f621e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.892072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed08f0-e6b8-4eb3-958f-a7f4b57f621e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.892355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed08f0-e6b8-4eb3-958f-a7f4b57f621e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.893957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 75/148] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:55:59.894890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267879-8fd7-42e1-b54a-20018ec629b4 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.895151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267879-8fd7-42e1-b54a-20018ec629b4 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.895430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99267879-8fd7-42e1-b54a-20018ec629b4 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.900888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 76/149] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:55:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:55:59.902317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.902605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.902886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.928503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:55:59.929836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:55:59.932321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:55:59.933382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.933676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.933959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/7e6273b9f6c24071abf2853f89ab26dc/roles/6d2b1080d0d54e43a997657c5492ad28` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:55:59.939656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, user_id=7e6273b9f6c24071abf2853f89ab26dc, role_id=6d2b1080d0d54e43a997657c5492ad28)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:55:59.941267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3357b7-ae44-4c1e-a402-2dbc70ab499b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:55:59.967834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 74/150] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:55:59 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/users/7e6273b9f6c24071abf2853f89ab26dc/roles/6d2b1080d0d54e43a997657c5492ad28 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:55:59.969203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:55:59.969498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:55:59.969783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.013955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:00.015608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:00.064820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.078676 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 77/151] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:00.118225 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 75/152] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:00.119765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.135410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.165715 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.338801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75ed15-48ca-4733-95d3-7c2343286540 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.394319 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b458e-4920-49ef-800b-b36fcdf45718 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.412316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.414193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.414698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.415219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.439639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:00.455136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:00.456239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:00.457202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:00.464795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 78/153] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:00.468990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.473770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.475066 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.476946 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.477705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.501965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:00.512268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:00.513361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:00.514302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:00.524441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 76/154] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:00.527588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.572801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.611122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.666646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6465c26-4d98-4e48-b505-2a7d58f471bc None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.717154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1566afe-f237-44ab-b101-07641b61cebe None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.725320 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.726014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.726339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.726623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.727396 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:00.727709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:00.729213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:00.744421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 79/155] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:00.750090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.785571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.786347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.786754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.787040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.787927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:00.788243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:00.791053 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:00.805493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 77/156] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:00.810850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:00.906541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12a59-42a3-4a8b-9c47-ab6cb77003d1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.932590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c84f6-9d3b-41f6-a619-06b1fa8446c5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:00.940824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:00.972327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:00.973308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.973610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.973890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.975119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:00.976827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0834-27ad-4135-bf1d-c455ead8ed4c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:00.978857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 78/157] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:00.980367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8f04f-2a98-4568-b776-93d67deba3fb None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:00.980741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8f04f-2a98-4568-b776-93d67deba3fb None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:00.981090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8f04f-2a98-4568-b776-93d67deba3fb None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:00.983344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 79/158] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:00.987850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8f04f-2a98-4568-b776-93d67deba3fb None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:01.017261 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:01.018049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:01.018332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:01.018599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:01.024524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:01.026019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088adca-3e69-40da-b3a6-e2628e723d4a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:01.028022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 80/159] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:01.029277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:01.029562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:01.029839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:01.050606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:01.089495 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:01.090722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:01.155886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8f04f-2a98-4568-b776-93d67deba3fb None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:01.217176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:01.217855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:01.218142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:01.218421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:01.239267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 81/160] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:01.240537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:01.252654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:01.321702 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:01.328988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:01.439729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 80/161] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:01.440914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:01.460596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fbab91-ae83-420f-90b5-e0fb10b6de9a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:01.502282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:01.551941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:01.564637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:01.565306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:01.565592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:01.565873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:01.570936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ea8d-0b9c-49f9-8b78-fecd6d21001b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:01.633774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:01.634597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:01.634881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:01.635158 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:01.650684 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:01.667154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:01.668259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:01.669213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 81/162] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:01.673258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:01.710702 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:01.712242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:01.872833 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 82/163] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:01.873618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:01.928782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:01.966432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b1f35-d6f3-40be-9f79-71290aeab6f8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:01.994091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:02.045748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3874-a418-4fe6-9387-473f43540bc5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.057258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44529c43-488f-4919-81da-bd780db65359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.058156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44529c43-488f-4919-81da-bd780db65359 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.058441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44529c43-488f-4919-81da-bd780db65359 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.058721 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44529c43-488f-4919-81da-bd780db65359 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.069251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44529c43-488f-4919-81da-bd780db65359 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.071612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44529c43-488f-4919-81da-bd780db65359 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.073785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 82/164] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:01 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.074587 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44529c43-488f-4919-81da-bd780db65359 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.094854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.095570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.095863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.096155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:02.097308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.105854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.114974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 83/165] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.115824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.242747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44529c43-488f-4919-81da-bd780db65359 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.281826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:02.295268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1c794-b072-474d-a3aa-b6372f83c518 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.308238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.309161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.309449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.309730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.317188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.318656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.321479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 83/166] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.322082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.356665 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.357354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.357643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.357926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.358782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:02.359099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.360630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.367758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 84/167] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=afa620995009427092fc6164c9cc643b => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.368648 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.420224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:02.519016 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd732c4c-853b-4f3a-a350-162e47b205a2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.541236 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be1a99-bff6-425e-a42b-c8a4126507a9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.556516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.557180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.557472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.557760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.558644 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:02.558963 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.560501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a67696-c4ea-42bc-8a27-7ad0a2497b58 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.567478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 85/168] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=dd3a79631b06401fab3c7670acb9024d => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.568871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce0d-ed4d-43fd-bd06-9a169be55d00 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.569164 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce0d-ed4d-43fd-bd06-9a169be55d00 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.569448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2ce0d-ed4d-43fd-bd06-9a169be55d00 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.571621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 86/169] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:02.577760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.578313 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.578779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.606289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:02.607950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.608156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:02.609259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.609703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.610105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] PATH_INFO: `/v3/users/af21adf4b88843c7988a1c2761d5763b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:02.616435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] RBAC: Authorizing `identity:get_user(user_id=af21adf4b88843c7988a1c2761d5763b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:02.616435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:02.621454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 84/170] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/users/af21adf4b88843c7988a1c2761d5763b => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:02.628421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.654643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:02.791425 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 87/171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:02.834975 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08157fa7-5952-4b5e-a3fe-a86e232d700e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:02.848624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:02.860472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:02.894079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:02.894926 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:02.895228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:02.895512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.018723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:03.020234 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:03.096574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d6e97-70a1-49b3-bdbf-2a1add694d06 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:03.155054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:03.155984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.156274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.156652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.185625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:03.204427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:03.205502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:03.206391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 88/172] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:03.213346 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:03.219503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 85/173] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:56:03.226124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:03.257071 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:03.287094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:03.536432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca61d6-c452-4cfb-abd0-0a2f3e885c7b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:03.609292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:03.610208 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.610505 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.610785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.612164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:03.613665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:03.615576 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 86/174] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:03.625440 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:03.696645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efccc73-3ddf-42dc-b392-a3502703f6d5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:03.752810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:03.770908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1478fa-b9f3-4b68-b569-513db91411b4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:03.810886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:03.811593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.812001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.812294 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.813384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=af21adf4b88843c7988a1c2761d5763b) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:03.813703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:03.815645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1b452-4199-436b-bb8c-e57695a7924f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:03.819541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:03.820433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.820654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.823011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.823011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:03.823011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:03.823410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:03.823803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 87/175] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/role_assignments?user.id=af21adf4b88843c7988a1c2761d5763b&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:03.825279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370c2f-75fd-40fe-aba1-64e4eebb7fc1 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.825574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370c2f-75fd-40fe-aba1-64e4eebb7fc1 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.825855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370c2f-75fd-40fe-aba1-64e4eebb7fc1 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.827110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 88/176] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:03.833508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:03.834052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:03.834699 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:03.854034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 89/177] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 641 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:03.860285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:03.869553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:03.871713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:03.956622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 89/178] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:03.974887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.059412 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90a797-61b2-40a3-b9f2-ed5c856e1f2b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.076041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.106386 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.106646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68727d8-0c4f-4fd9-b28c-763e237eb9ff None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.107282 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.107565 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.107843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] PATH_INFO: `/v3/users/4e2602f304894d35a33730dfcd6cc0cc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.109125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] RBAC: Authorizing `identity:get_user(user_id=4e2602f304894d35a33730dfcd6cc0cc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.110619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efdd3f-96f1-4cf1-8a9f-361f9d41fbbf None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.112728 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 90/179] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/users/4e2602f304894d35a33730dfcd6cc0cc => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:04.114087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d47d9-45b1-494d-b05b-d905e09ef282 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.114377 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d47d9-45b1-494d-b05b-d905e09ef282 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.114655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d47d9-45b1-494d-b05b-d905e09ef282 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.115958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 91/180] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:04.117279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68b538-8fbe-4815-9c37-47a42a62188b None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.117626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68b538-8fbe-4815-9c37-47a42a62188b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.117898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68b538-8fbe-4815-9c37-47a42a62188b None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.119594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 92/181] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:04.120387 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68b538-8fbe-4815-9c37-47a42a62188b None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.158079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.158927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.159210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.160251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.174744 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-9a5f0389-90b4-475d-9418-e6941adee77d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:04.184056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:04.185235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:04.186167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:04.192200 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 90/182] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:04.192392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd90d-8f1c-4a39-ad1c-14d40ec19189 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.193922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd90d-8f1c-4a39-ad1c-14d40ec19189 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.193922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd90d-8f1c-4a39-ad1c-14d40ec19189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.219738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-62afd90d-8f1c-4a39-ad1c-14d40ec19189 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:04.221418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-62afd90d-8f1c-4a39-ad1c-14d40ec19189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:04.261422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68b538-8fbe-4815-9c37-47a42a62188b None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.301734 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.302390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.302674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.302948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.306109 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 91/183] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:04.307265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.307664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.307950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.324279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.325832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:04.327185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:04.400505 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.411187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.411187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.412591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 92/184] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:04.415726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.536068 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2feda1-1149-437e-a623-51c6e50124ed None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.540210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 93/185] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:04.541233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.585127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.586670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.587153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.587645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.588668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:04.591623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.591623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.594038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.601085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 93/186] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:04.602668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.659472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3485332d-65fe-4279-844b-27e9dec6daa1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.676293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.696124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.696991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.697273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.697551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.698973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.700384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.702125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 94/187] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:04.702958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.719762 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7897bf-2c4a-4ea7-a475-d2b6e8989323 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.754717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.755549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.755844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.756126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.768909 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-69eb52f5-d85d-4afa-b8af-476dcafb5075 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:04.777248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:04.778456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 94/188] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:04.779825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381cc38-32ac-4a8c-abf3-610aa08697f0 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.780841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381cc38-32ac-4a8c-abf3-610aa08697f0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.780841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381cc38-32ac-4a8c-abf3-610aa08697f0 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.781552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 95/189] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:04.784574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381cc38-32ac-4a8c-abf3-610aa08697f0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.819524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415ca51-de51-4c47-95d0-583813a3702b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.830583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.856072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.856984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.857274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.857555 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] PATH_INFO: `/v3/projects/2415b8fb72a74388ba4695e700a86a3e` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.858771 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] RBAC: Authorizing `identity:get_project(project_id=2415b8fb72a74388ba4695e700a86a3e)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.860448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.862275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 95/190] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/projects/2415b8fb72a74388ba4695e700a86a3e => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:04.863052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.894354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5381cc38-32ac-4a8c-abf3-610aa08697f0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:04.923324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:04.935885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:04.936824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:04.937110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:04.937393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:04.952303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:04.953819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:04.955547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:04.957825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:04.971678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959d7ab-3bff-4e57-ba86-9f2b2817001d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.007867 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.008795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.009050 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.011934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.011934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.011934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff364a-aaef-4744-99bb-18377c7ad906 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.042405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 96/192] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:04 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:05.042405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.042405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.042405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.055933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:05.057321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:05.077843 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449aa44c-c896-4cc7-9156-15d913675a36 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.091877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.131048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.131645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.132938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.133263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.134067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:05.134417 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.135836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.150435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 97/193] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:05.150435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.150739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 97/194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:05.150739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.213465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.291546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f7ca5-92fc-4332-a3d5-d20f0abef9dd None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.291964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69f305-cc3c-4b3a-a5a8-e633abd0eb83 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.341657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.342330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.342618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.342736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.342935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.343322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.343597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.343772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=2415b8fb72a74388ba4695e700a86a3e, user.id=4e2602f304894d35a33730dfcd6cc0cc) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:05.343878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.344054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.345588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.356284 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 98/195] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/role_assignments?user.id=4e2602f304894d35a33730dfcd6cc0cc&scope.project.id=2415b8fb72a74388ba4695e700a86a3e&role.id=afa620995009427092fc6164c9cc643b => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:05.357190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.400348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.457161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.460021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.505412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a341088-7eb6-4d5d-bdf4-f77e878b8090 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.514622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.551714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.552856 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.553249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.553663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.555773 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.556885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.559084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 99/196] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:05.560136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.592992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 98/197] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:05.594331 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.666947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.684204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb24b4-3d3a-4dde-a2ad-dbe4f190222e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.719156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1338e-eb88-45f0-9c79-f95249bc591c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.726824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.727824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.728441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.728441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.740085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:05.749122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:05.750536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 100/198] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:05.755250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.772109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.774165 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.774670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.775155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.795130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.856289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.858541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.875152 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42218552-628f-4079-a9a4-b30a039c74a6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:05.917610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:05.918095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:05.918382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:05.918664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:05.919968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:05.921615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:05.923620 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 101/199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:05.924661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:05.935465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:05.987107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 99/200] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:05.988602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.051952 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d59b6-acc0-474f-a14b-3da23ccc8e7d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.056945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.092945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.093714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.093948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.094197 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.094985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:06.095282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.096732 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.100814 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b45d2-5bdc-4dfc-91ad-4b3b343709c9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.106881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 102/201] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.112300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.151975 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.153093 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.153588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.154086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.155142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=a2cc29e8684545d68e9e4850a7a4e31f) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:06.156175 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.158705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.168190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 100/202] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:05 2022] GET /identity/v3/role_assignments?user.id=a2cc29e8684545d68e9e4850a7a4e31f&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.170292 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.208777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.286199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd037485-a0af-414b-b322-f54670f7e6a5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.341025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23edb14-da9b-46a5-a896-2d54a1cff551 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.385049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.386699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.387193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.387676 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] PATH_INFO: `/v3/users/7e6273b9f6c24071abf2853f89ab26dc` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.389329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] RBAC: Authorizing `identity:get_user(user_id=7e6273b9f6c24071abf2853f89ab26dc)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.389860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.390741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.391032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.391314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.392107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.392619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.394094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.394583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 101/203] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/users/7e6273b9f6c24071abf2853f89ab26dc => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.399963 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 103/204] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.401606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.405520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.445463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.515351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.523733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c576a19-46c5-42bb-b236-972947356056 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.566620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.568776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.570046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.571951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.595298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6b9f2-7fdb-41b1-ba4c-114446523227 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.678375 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.680849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.686673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.687876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.688421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.688968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.690686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=a2cc29e8684545d68e9e4850a7a4e31f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:06.691270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.702390 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.724190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 104/205] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/role_assignments?user.id=a2cc29e8684545d68e9e4850a7a4e31f&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=dd3a79631b06401fab3c7670acb9024d => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.796828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.810490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 102/206] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:56:06.812295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:06.928486 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:06.935852 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28722ea-e3e6-46c8-a42f-b15fcfc372ba None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:06.984246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:06.985144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:06.985440 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:06.985719 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:06.987023 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:06.988592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b17a-08a8-425a-b367-e561449bff1e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:06.990363 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 105/207] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:06.997395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:07.130507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1689c0d-463c-47b7-a049-55c1cf670f4c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:07.173473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:07.174524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:07.175046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:07.175529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:07.176613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, scope.project.id=1557b854d9814d43a2958ac6f6f85e48, user.id=7e6273b9f6c24071abf2853f89ab26dc) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:07.177157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:07.178893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a750f-7e05-4802-b78c-6cb6d04bf819 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:07.191447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 103/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:06 2022] GET /identity/v3/role_assignments?user.id=7e6273b9f6c24071abf2853f89ab26dc&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:07.944378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867a2f-4f73-4558-8813-f98c2507674e None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:07.945472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867a2f-4f73-4558-8813-f98c2507674e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:07.945558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867a2f-4f73-4558-8813-f98c2507674e None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:07.947838 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 106/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:07.954147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:07.954447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:07.954713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:07.961580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553980-68d9-46a0-b63d-c653c7e62704 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:07.961878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553980-68d9-46a0-b63d-c653c7e62704 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:07.962149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553980-68d9-46a0-b63d-c653c7e62704 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:07.963547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 107/210] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:07.969082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:07.969455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:07.969749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:07.975056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:07.976372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:07.987493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:07.988820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:08.045602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.053226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 104/211] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:08.085376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 108/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:08.093367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.117416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.230376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9dcd6e-c31e-44d5-abc6-3908ac6241b7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.254263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.267500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe7825-7133-48a7-ad9f-e334bca98148 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.271305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.272205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.272601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.273074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.284834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:08.293683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:08.294974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:08.296119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:08.297051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 105/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:08.303458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.316507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.317630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.318107 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.318547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.321000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.322679 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.324599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 109/214] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:08.331799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.375501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.435371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893312d-88ae-4dda-8a4f-33056df8fa56 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.456228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d537f-d65a-4dbe-a05b-bb66da98eaab None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.475397 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.476734 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.477276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.477838 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.478823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:08.479349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.481177 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.491418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 106/215] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:08.495139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.495819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.496105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.496383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.497117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.497709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.498581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.528503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 110/216] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:08 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:08.630395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b845f-86fd-4cec-8871-3ecc367ea8b1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.653849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.679580 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.680802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.681152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.681444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] PATH_INFO: `/v3/users/e23386b6bbf040a9a365fa037607a718` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.700090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] RBAC: Authorizing `identity:get_user(user_id=e23386b6bbf040a9a365fa037607a718)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.701687 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8eeb8-8acc-4f78-857b-480bd2f37a2d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.704045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 107/217] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/users/e23386b6bbf040a9a365fa037607a718 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:08.709819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:08.733448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:08.827468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e443bb-076e-48de-8511-764fff46e4fd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:08.865391 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:08.866062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.866342 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.866651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.895865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5daa-eaa9-4a02-9434-c5bd7c4acec8 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.896472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5daa-eaa9-4a02-9434-c5bd7c4acec8 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.896927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c5daa-eaa9-4a02-9434-c5bd7c4acec8 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.899413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 108/218] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:08.905245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:08.905550 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:08.905815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:08.926094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:08.927637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:08.968212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:08.970556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:08.998160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.028963 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 109/219] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:09.052623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.125126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 111/220] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:09.137808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.201525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.204833 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92bd09-0481-4ee6-8e96-1acbbd551c75 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.241412 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.242774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.243060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.243350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.257044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:09.266537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:09.267640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:09.268570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:09.273111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 110/221] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:09.275085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.400428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6892b-4d53-4bcf-bc55-d2376cead4bb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.400642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598616d-c1c5-47c8-ad41-7a5e2155fea2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.434284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798aed-3426-4e75-b633-be50187c6add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.435085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798aed-3426-4e75-b633-be50187c6add None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.435368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798aed-3426-4e75-b633-be50187c6add None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.435646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798aed-3426-4e75-b633-be50187c6add None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.436433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798aed-3426-4e75-b633-be50187c6add None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:09.436763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798aed-3426-4e75-b633-be50187c6add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:09.438163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798aed-3426-4e75-b633-be50187c6add None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:09.447663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 111/222] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:09.454083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798aed-3426-4e75-b633-be50187c6add None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.455161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.493284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.504974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.507704 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.508256 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.508569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.513795 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:09.515846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:09.522119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 112/223] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:09.527072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.629355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798aed-3426-4e75-b633-be50187c6add None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.661061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3494d6-fe2b-4ac7-bc4b-0e58ac232926 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.668051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.669005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.669374 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.669757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.671617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:09.673527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783ac22-de15-44a0-b347-be5ad0a0c97f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:09.675440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 112/224] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:09.676587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52572764-a599-4ec5-9e26-1810f118308a None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.676876 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52572764-a599-4ec5-9e26-1810f118308a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.677156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52572764-a599-4ec5-9e26-1810f118308a None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.678417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 113/225] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:09.683877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52572764-a599-4ec5-9e26-1810f118308a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.707352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.708644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.708934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.709211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.710069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=e23386b6bbf040a9a365fa037607a718) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:09.711004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:09.713798 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f41929-47ba-4a83-b906-580d666d195b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:09.725316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 113/226] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/role_assignments?user.id=e23386b6bbf040a9a365fa037607a718&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=afa620995009427092fc6164c9cc643b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:09.725316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.725316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.725316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.764199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:09.765421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:09.861435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.885844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:09.889826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52572764-a599-4ec5-9e26-1810f118308a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:09.936059 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 114/227] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:09.951120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:09.952066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:09.952389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:09.952752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:09.954079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:09.955680 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:09.957626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 114/228] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:09.964100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:09.974351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:10.107782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46819f6-9adc-4879-b8cb-153b98bb6ea9 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:10.159856 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:10.160863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:10.161148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:10.161467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/users/a2cc29e8684545d68e9e4850a7a4e31f/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:10.166199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, user_id=a2cc29e8684545d68e9e4850a7a4e31f, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:10.167727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:10.174302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64e09f-8333-47d9-918a-04b4d0bf5e01 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:10.196393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 115/229] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:56:09 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/users/a2cc29e8684545d68e9e4850a7a4e31f/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:10.234484 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:10.235326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:10.235611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:10.235887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:10.249619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:10.258241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:10.260614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:10.262758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 115/230] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:10.268391 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:10.351504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:10.438851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7330325-91a0-49bf-b114-157f21461f71 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:10.479951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:10.480665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:10.480947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:10.481222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:10.481977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:10.482282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:10.483662 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:10.493346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 116/231] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:10.499176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:10.522545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:10.622648 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9d7d-0d8a-41fe-95f2-cfc2dde7e3ad None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:10.670385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:10.671269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:10.671553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:10.671829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:10.677195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:10.678684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:10.680685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 116/232] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:10 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:10.686303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:10.823392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba47d511-343e-4d48-af3a-30ca5805b4d9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:10.825009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:10.876759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:10.877630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:10.877912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:10.878188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:10.881133 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:10.883358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bff214-0992-4203-b76b-2082e64bb0f9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:10.885330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 117/233] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:10 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:10.890762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:11.020985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:11.044236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a6970-0255-47e6-8d3f-41ca2f16641a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:11.098361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:11.099730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.100162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.100666 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/users/a2cc29e8684545d68e9e4850a7a4e31f/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.107109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, user_id=a2cc29e8684545d68e9e4850a7a4e31f, role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:11.109475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015ea08-75e1-4cd9-815b-9a33e21f83ca None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:11.139090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 117/234] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:56:10 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/users/a2cc29e8684545d68e9e4850a7a4e31f/roles/dd3a79631b06401fab3c7670acb9024d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:11.375785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e560-5b8a-4a58-8611-cf718f859c72 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.376315 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e560-5b8a-4a58-8611-cf718f859c72 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.376315 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484e560-5b8a-4a58-8611-cf718f859c72 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.378251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 118/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:11.385907 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c2930-0320-4114-8f49-23785927ecef None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.386647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c2930-0320-4114-8f49-23785927ecef None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.386647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c2930-0320-4114-8f49-23785927ecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.415123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-c65c2930-0320-4114-8f49-23785927ecef None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:11.416478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-c65c2930-0320-4114-8f49-23785927ecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:11.550775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 118/236] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:11.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e560-5b8a-4a58-8611-cf718f859c72 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:11.656173 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95601e-7640-4101-8413-69519e113b93 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.656859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95601e-7640-4101-8413-69519e113b93 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.657340 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95601e-7640-4101-8413-69519e113b93 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.658994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 119/237] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:11.665180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.665468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.665748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.698903 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:11.700185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:11.707162 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484e560-5b8a-4a58-8611-cf718f859c72 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:11.743798 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:11.744638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.744920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.745204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.757820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:11.766094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:11.767131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:11.768052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 119/238] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:11.774285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:11.816766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:11.868118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 120/239] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:11.904048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:11.915963 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:11.921275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b0fe3-9578-48a5-819e-f9e7e7e76fcc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:11.960169 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:11.961270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:11.961482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:11.961731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:11.962504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:11.962818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:11.964220 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:11.973822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 120/240] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:11.989506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:12.114438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54426f0-1dc0-44ab-8005-7c2311548394 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:12.188738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:12.194624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:12.196138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.196449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.196733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.202117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:12.203619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:12.205496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 121/241] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:12.208700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8461-7dc0-4c74-8ffe-83a77e7c4d52 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:12.213406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:12.254105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:12.256085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.256373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.256679 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] PATH_INFO: `/v3/users/465d7402e0504d10ac36635a6c0fea38` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.289850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=465d7402e0504d10ac36635a6c0fea38)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:12.291078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:12.292900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 121/242] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:11 2022] GET /identity/v3/users/465d7402e0504d10ac36635a6c0fea38 => generated 297 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:12.306355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:12.368881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:12.419822 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04d098-443a-4e56-aabf-31604b9f7b91 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:12.486944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9b6f3-b28c-450d-99ab-0a793c9f0ac9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:12.507531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:12.507531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.507531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.508367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.508651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:12.510060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:12.527007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:12.527601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.527921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.528262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.553677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:12.557756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 122/243] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:12 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:12.623048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:12.624605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee1995b-a8a9-44a8-8ada-4ab8881f0956 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:12.751211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 122/244] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:56:12.756587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:12.831627 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05dc1-7e70-4c9c-98f4-de0c1946c3fc None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.831967 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05dc1-7e70-4c9c-98f4-de0c1946c3fc None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.834723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05dc1-7e70-4c9c-98f4-de0c1946c3fc None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.836925 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 123/245] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:12.846817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a790ee-996b-4917-9931-35e117c67afc None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.847249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a790ee-996b-4917-9931-35e117c67afc None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.847576 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a790ee-996b-4917-9931-35e117c67afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.872747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-22a790ee-996b-4917-9931-35e117c67afc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:12.873967 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-22a790ee-996b-4917-9931-35e117c67afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:12.920565 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4477828-9fb5-4cdf-b2d8-a28197d18e93 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:12.959245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:12.962916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:12.975744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:12.978298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:12.978298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:12.978298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:12.979464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:12.981094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:12.984000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 123/246] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:12 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:12.990056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 124/247] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:12.993452 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.018586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a790ee-996b-4917-9931-35e117c67afc None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.144328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d001c8-7f81-49c2-a4b6-97232e1b838e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.144680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a790ee-996b-4917-9931-35e117c67afc None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.201465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.202792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.203078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.203351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.207255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.207941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.208226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.208530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.209382 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=465d7402e0504d10ac36635a6c0fea38) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:13.209704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:13.211301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca6640a-8ba3-4a14-9a05-2b0162665030 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:13.224090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:13.224566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:12 2022] GET /identity/v3/role_assignments?user.id=465d7402e0504d10ac36635a6c0fea38&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:13.226677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae5c6-4c70-4c32-8dc7-b90c377082f2 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.226968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae5c6-4c70-4c32-8dc7-b90c377082f2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.227241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae5c6-4c70-4c32-8dc7-b90c377082f2 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.229019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 125/249] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:13.233932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:13.235030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:13.235940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:13.236655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.236655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.236655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.236871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 125/250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:13.244427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.274942 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:13.276070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:13.331681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:13.397548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:13.419970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 126/251] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:13.437023 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4016ab0-5dc2-442c-b2cc-88d83658a667 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.455049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.490469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.491166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.491431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.491747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.493424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:13.493750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:13.495445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:13.506134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 126/252] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:13.513496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.690386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e585-c952-498c-9c4f-d3d7bf31604b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.692149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:13.728858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4d63-4825-45cd-8101-43d0c9cbb78a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.756059 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.757924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.758448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.758961 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] PATH_INFO: `/v3/users/e23386b6bbf040a9a365fa037607a718` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.761416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] RBAC: Authorizing `identity:get_user(user_id=e23386b6bbf040a9a365fa037607a718)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:13.762912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:13.764991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 127/253] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/users/e23386b6bbf040a9a365fa037607a718 => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:13.770398 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.809746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.810482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.810769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.811047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.839696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:13.847629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:13.848701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:13.849623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:13.850333 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 127/254] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:13.857276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:13.933373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30233b2-0462-4baa-a456-3899219651a1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:13.964807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:13.975113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:13.976416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:13.976824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:13.977212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:13.978587 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:13.980692 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:13.983739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 128/255] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:13.990252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:14.079385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24cdf6-fe0f-4aca-975f-cf7e0c482fc6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:14.112886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:14.182109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:14.183088 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.183526 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.183954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.184892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:14.185371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:14.186929 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:14.214802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 128/256] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:14.220526 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:14.221250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8a51-6772-4314-8032-7f79c3ca5617 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:14.270136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:14.271026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.271322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.271601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/users/e23386b6bbf040a9a365fa037607a718/roles/afa620995009427092fc6164c9cc643b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.274682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, user_id=e23386b6bbf040a9a365fa037607a718, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:14.276212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fa633-7a6b-426d-a554-f381dd8796b9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:14.298089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 129/257] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:56:13 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/users/e23386b6bbf040a9a365fa037607a718/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:14.416892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b3ea5-8883-4ba2-b80c-6f6b05685825 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:14.428632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269abda7-2471-4dda-8b6a-0cc0d531217c None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.429128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269abda7-2471-4dda-8b6a-0cc0d531217c None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.429516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269abda7-2471-4dda-8b6a-0cc0d531217c None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.432543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 130/258] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:14.437237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.437797 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.438180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.439203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:14.470901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:14.472151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:14.492640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:14.493565 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.493850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.494126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.496945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:14.498421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:14.501581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 129/259] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:14 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:14.512444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:14.645786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 131/260] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:14.672514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:14.712681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:14.770659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381557c-2697-481d-8720-a229b113fc7f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:14.820087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:14.820781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.821066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.821344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:14.875171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:14.884151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f45880-7b01-4fef-8eea-9767dfed5aa9 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:14.897890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:14.899383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:14.981389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:14.982167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:14.982458 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:14.982739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.013467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:15.014133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 130/261] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:15.022028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:15.023698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:15.024782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:15.026415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.033582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.250298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:15.284678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:15.302013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2161b-9f91-40f6-b4a1-37fda1f1ac7c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.319278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b35060-ddd9-44ac-860b-8a04d7651401 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.381139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.381941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.382811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.383096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.383863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:15.384177 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:15.385761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:15.400159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 133/263] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:15.402238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.403142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.403433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.403717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.404964 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:15.405781 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.406668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:15.408429 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 131/264] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:15.414933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.524078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d457b-72ea-4151-8743-0e23df5036d7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.564907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.566159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.566652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.567082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] PATH_INFO: `/v3/users/a2cc29e8684545d68e9e4850a7a4e31f` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.568618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] RBAC: Authorizing `identity:get_user(user_id=a2cc29e8684545d68e9e4850a7a4e31f)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:15.570472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:15.576740 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 134/265] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/users/a2cc29e8684545d68e9e4850a7a4e31f => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:15.578498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:15.600215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:15.640213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906515f7-7885-41ea-a8cf-926624990fcc None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.701017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfadab-5582-46b8-baee-5907d8142c59 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:15.721152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.722001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.722383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.722765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.723713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=a2cc29e8684545d68e9e4850a7a4e31f) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:15.724128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:15.725743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:15.742654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:15.743604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:15.743722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 132/266] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/role_assignments?user.id=a2cc29e8684545d68e9e4850a7a4e31f&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:15.744231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:15.744771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:15.831011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:15.832847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:15.853830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:15.965787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 135/267] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:56:15.972019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.000737 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:16.141702 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322928b1-ebef-46c8-b5b3-2f1caf3aaf6c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:16.193177 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:16.194045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.194327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.194777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.197738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:16.199213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebaee5-7a2a-47de-8cc3-298f56b97213 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:16.200981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 133/268] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:15 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:16.207221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.245040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cfb17-06bf-4d4c-99b2-cd824e7cfed4 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.245381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cfb17-06bf-4d4c-99b2-cd824e7cfed4 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.245607 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cfb17-06bf-4d4c-99b2-cd824e7cfed4 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.246995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 134/269] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:16.253575 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.253893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.254119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.287830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:16.289602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:16.420258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645dffb-6848-4d8f-b753-422b3e3032cb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:16.436297 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 135/270] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:16.461140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:16.470812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.497028 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:16.502610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:16.502610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.502610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.502610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.503386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=a2cc29e8684545d68e9e4850a7a4e31f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:16.503386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:16.504093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e00ba-0553-4fd3-95a9-bb1ae1e2032a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:16.520443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 136/271] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:16 2022] GET /identity/v3/role_assignments?user.id=a2cc29e8684545d68e9e4850a7a4e31f&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=dd3a79631b06401fab3c7670acb9024d => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:16.602070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ac7c-542e-4881-9812-f0d8377d9785 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.602754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ac7c-542e-4881-9812-f0d8377d9785 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.603255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ac7c-542e-4881-9812-f0d8377d9785 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.605326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 137/272] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:16.610593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d6840-4500-4964-960c-caf7b9a5d7f3 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.610902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d6840-4500-4964-960c-caf7b9a5d7f3 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.611164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d6840-4500-4964-960c-caf7b9a5d7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.649657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-324d6840-4500-4964-960c-caf7b9a5d7f3 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:16.650945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-324d6840-4500-4964-960c-caf7b9a5d7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:16.660819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28126dcd-6ef0-4969-9897-a25ba40552ad None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:16.704125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:16.705243 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.705694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.706133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.721247 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-25e65fa0-b4f2-46f1-98f5-1ffe77d4c96f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:16.731650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:16.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:16.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:16.735365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 136/273] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:16.735365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb527-ef73-4ea4-bfb9-a3711ece6450 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.735723 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb527-ef73-4ea4-bfb9-a3711ece6450 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.735985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fb527-ef73-4ea4-bfb9-a3711ece6450 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.737632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 137/274] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:16.743750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb527-ef73-4ea4-bfb9-a3711ece6450 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.753474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 138/275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:16.755586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.756097 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.756634 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.774119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:16.775689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:16.831090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:16.860476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 139/276] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:16.862069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.872262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fb527-ef73-4ea4-bfb9-a3711ece6450 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:16.913644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:16.919203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:16.919881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:16.920162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:16.923572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:16.923572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:16.923572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:16.923572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:16.937789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 138/277] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:16.939204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:16.985239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89903271-aff1-45d2-a8b0-78157beac22f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.025449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.026426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.026717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.027022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.038040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:17.046670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:17.048018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:17.048996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 140/278] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:17.050765 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.074413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c48f4-4cee-44a6-b1dc-f33010be0f23 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.113359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.114595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.114890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.115183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.116467 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.118484 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.124901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 139/279] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.125758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.128911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.171971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ea7fd-db10-48b0-ab73-8be14828421e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.173849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.210567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.211997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.212290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.212605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] PATH_INFO: `/v3/users/9b83a17e264141c6a848a459b26a5ad8` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.228584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] RBAC: Authorizing `identity:get_user(user_id=9b83a17e264141c6a848a459b26a5ad8)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.230097 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.231869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 141/280] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/users/9b83a17e264141c6a848a459b26a5ad8 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.232778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.261632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26057ba-dce5-425a-928e-f992f38751a3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.301372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.302155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.302446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.302728 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.303585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:17.303902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.306314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.316371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 140/281] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.317270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.362698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4aeb8-4287-468c-a853-a21d97378bde None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.392646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.401976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.402655 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.402943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.403230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.404135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.405610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.431567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 142/282] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:17 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:17.432855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.455936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.460667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9481bf-6d2c-4733-b021-66016d350c86 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.519950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.520919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.521280 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.521695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.522739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, user.id=9b83a17e264141c6a848a459b26a5ad8, scope.system=all) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:17.523292 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.525142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d487b3-dbeb-4b3d-b3f1-277b353fe3b6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.532234 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 141/283] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/role_assignments?user.id=9b83a17e264141c6a848a459b26a5ad8&scope.system=all&role.id=afa620995009427092fc6164c9cc643b => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.592755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362cea4-e8ca-407a-9f2c-225e3ed8de52 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.628544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa728b2-341a-4466-8b82-daa00611bf64 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.628839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa728b2-341a-4466-8b82-daa00611bf64 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.629132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa728b2-341a-4466-8b82-daa00611bf64 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.631076 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 142/284] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:17.637774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.638062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.638342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.640610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.641944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.642474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.642922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] PATH_INFO: `/v3/users/465d7402e0504d10ac36635a6c0fea38` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.644544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] RBAC: Authorizing `identity:get_user(user_id=465d7402e0504d10ac36635a6c0fea38)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.646358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.649179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 143/285] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/users/465d7402e0504d10ac36635a6c0fea38 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.657746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.658775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:17.659957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:17.775134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 143/286] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:17.798644 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.799215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.819032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af10f4-ece9-4865-9bda-7fcc2b2495be None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.873839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.874716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.875001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.875282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:17.876568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:17.878034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:17.879938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 144/287] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:17.886458 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:17.892293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:17.940664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c7382-8bc9-4dd2-bfaf-c3bcc7c4a553 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:17.988388 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:17.989265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:17.989573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:17.989852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.010814 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:18.019304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:18.020367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:18.021301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 144/288] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:18.027455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:18.057090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092eb03-3463-4257-bc81-68f6dc5fd7bd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:18.123508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:18.124431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:18.124711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:18.124987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/users/465d7402e0504d10ac36635a6c0fea38/roles/6d2b1080d0d54e43a997657c5492ad28` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.128170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, user_id=465d7402e0504d10ac36635a6c0fea38, role_id=6d2b1080d0d54e43a997657c5492ad28)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:18.129680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:18.154550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 145/289] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:56:17 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/users/465d7402e0504d10ac36635a6c0fea38/roles/6d2b1080d0d54e43a997657c5492ad28 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:56:18.193991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff420f-e28a-4e22-b2bf-788fe6c38183 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:18.215972 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:18.236296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:18.237020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:18.237306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:18.237593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.238379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:18.238690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:18.240131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:18.251032 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 145/290] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:18.256897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:18.274123 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:18.426055 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c55362-3fc3-4f20-a3fb-743aae949375 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:18.466769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:18.468169 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:18.468691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:18.469259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] PATH_INFO: `/v3/users/e23386b6bbf040a9a365fa037607a718` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.471249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] RBAC: Authorizing `identity:get_user(user_id=e23386b6bbf040a9a365fa037607a718)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:18.473010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:18.474922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 146/291] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:18 2022] GET /identity/v3/users/e23386b6bbf040a9a365fa037607a718 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:18.481140 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:18.592166 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5766742-1a77-4019-bfb2-d7f9362667d1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:18.627958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:18.628647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:18.628937 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:18.629201 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.692868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:18.705536 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:18.707030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:18.817688 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 146/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:18.823328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:18.918435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:18.940616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821254e3-7b1c-416a-87af-6e9151a1a020 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:18.984043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:18.984866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:18.985150 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:18.985433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:18.989362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:18.990848 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0254ac-3059-4569-a3d9-3261a9925e17 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:18.993450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 147/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:18 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:18.999174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:19.087992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:19.108608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9023d-a9a6-42eb-8c52-91b31003e2b1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:19.145346 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:19.146030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:19.146314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:19.146590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:19.147444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=e23386b6bbf040a9a365fa037607a718) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:19.147820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:19.150091 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fb70a-6a06-4a3b-846f-0e40548e0d5e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:19.157450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 147/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:18 2022] GET /identity/v3/role_assignments?user.id=e23386b6bbf040a9a365fa037607a718&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=afa620995009427092fc6164c9cc643b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:20.105172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310f68-5313-477e-94bf-0fcdfa2e602d None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.105712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310f68-5313-477e-94bf-0fcdfa2e602d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.105712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c310f68-5313-477e-94bf-0fcdfa2e602d None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.107388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 148/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:20.112980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc248577-8b30-49b8-9b93-6594c6f37da5 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.113399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc248577-8b30-49b8-9b93-6594c6f37da5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.113580 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc248577-8b30-49b8-9b93-6594c6f37da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.132499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-cc248577-8b30-49b8-9b93-6594c6f37da5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:20.133840 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-cc248577-8b30-49b8-9b93-6594c6f37da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:20.216553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 148/296] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:20.234279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c310f68-5313-477e-94bf-0fcdfa2e602d None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.352506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c310f68-5313-477e-94bf-0fcdfa2e602d None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:20.389024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:20.389875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.390161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.390436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.394297 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d61d6-78ec-4196-817f-364e7ce471af None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.394923 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d61d6-78ec-4196-817f-364e7ce471af None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.395417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d61d6-78ec-4196-817f-364e7ce471af None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.397539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 149/297] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:20.403205 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:20.404187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.404733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.405242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:20.411394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:20.412586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:20.414213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 149/298] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:20.421533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.430191 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:20.431755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:20.435256 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:20.532578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 150/299] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:20.558455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.562395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:20.563137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d757ee-10ee-435e-bcd2-772654a60677 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:20.613849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:20.614968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.615278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.615547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.617092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:20.617533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:20.619192 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:20.632544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 150/300] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:20.638271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.688225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7447528-8479-4310-9d48-bbe76bcb5364 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:20.729135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:20.730110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.730394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.730669 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.742826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:20.743045 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a127429b-5e6c-4cfd-bd49-92a6d41bca24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:20.750765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:20.751854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:20.753564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 151/301] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:20.754342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbe41-3011-4258-b765-57a32b442a17 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.754342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbe41-3011-4258-b765-57a32b442a17 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.754342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdbe41-3011-4258-b765-57a32b442a17 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.756010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 152/302] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:20.762157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdbe41-3011-4258-b765-57a32b442a17 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.787241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b45ccb-f9a2-47a3-9691-7a06ab3080c7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:20.837058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:20.837387 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:20.838691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.838691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.838905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] PATH_INFO: `/v3/users/9b83a17e264141c6a848a459b26a5ad8` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.840206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] RBAC: Authorizing `identity:get_user(user_id=9b83a17e264141c6a848a459b26a5ad8)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:20.841847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078e8f0-4f4e-4846-ba13-d83f121cf616 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:20.843552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/users/9b83a17e264141c6a848a459b26a5ad8 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:20.849517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.849813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.850101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.870555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:20.871778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:20.886522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdbe41-3011-4258-b765-57a32b442a17 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:20.927722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:20.928602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:20.929496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:20.930165 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:20.931225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:20.931545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:20.933452 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:20.942533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:20.945971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:20.953550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 152/305] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:20.965564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.041395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.074501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63ac79-af91-49f0-96e4-925dca237e55 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.076921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468adae-a5c4-4f79-bbb2-bceab94fc5d6 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.101880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.110999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.111871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.112164 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.112478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] PATH_INFO: `/v3/system/users/9b83a17e264141c6a848a459b26a5ad8/roles/afa620995009427092fc6164c9cc643b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.112705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.113587 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.113875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.113985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9b83a17e264141c6a848a459b26a5ad8, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.114334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] PATH_INFO: `/v3/users/111a0f7142194f949a55512c46091b05` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.115392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6d332-2915-42b4-98af-4717589de37e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.128846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] RBAC: Authorizing `identity:get_user(user_id=111a0f7142194f949a55512c46091b05)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.130537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.132503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 153/306] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:20 2022] GET /identity/v3/users/111a0f7142194f949a55512c46091b05 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:21.133489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 154/307] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 19:56:20 2022] PUT /identity/v3/system/users/9b83a17e264141c6a848a459b26a5ad8/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:21.133613 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.135226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3fca2-a707-4773-b3a8-d37d69da7d38 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.135226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3fca2-a707-4773-b3a8-d37d69da7d38 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.135226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3fca2-a707-4773-b3a8-d37d69da7d38 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.140463 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 155/308] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:21.142788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.143089 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.143372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.166259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:21.167511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:21.264908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a03228-5d22-4f72-91a7-b0cd5316528a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.267256 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 156/309] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:21.269475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.312865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.323454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.324276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.324598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.328463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.328463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.328463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.329481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 154/310] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:21.330325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.410540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054d0f2-1fbb-43d1-a533-cdbffe0afde9 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.439921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.448714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e653f6-843f-495b-85e9-bea5385f8ec9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.462249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.463563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.463848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.464454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.465434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, user.id=111a0f7142194f949a55512c46091b05, scope.system=all) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:21.465664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.467136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.473479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 157/311] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/role_assignments?user.id=111a0f7142194f949a55512c46091b05&scope.system=all&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:21.474079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.521230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.523732 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.524231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.524652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.546777 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:21.555913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:21.557282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:21.558404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:21.564462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 155/312] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:21.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.635211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584863a6-f746-4186-b098-8b2c04a231f4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.641301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.682810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.683663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.684570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.684570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] PATH_INFO: `/v3/groups` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.685329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.686983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.701648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bb69d-b595-4742-a08c-4474ff6f2357 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.706924 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 158/313] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 19:56:21 2022] POST /identity/v3/groups => generated 229 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:21.751318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.752201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.752618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.753015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.753985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:21.754397 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.755962 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.767524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 156/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:21.772773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:21.829791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:21.902031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f36186-f383-47c3-bb4b-e9cc4b42ebe5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:21.941997 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:21.943088 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:21.943514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:21.943890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] PATH_INFO: `/v3/users/465d7402e0504d10ac36635a6c0fea38` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:21.945890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] RBAC: Authorizing `identity:get_user(user_id=465d7402e0504d10ac36635a6c0fea38)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:21.947791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:21.950162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 159/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/users/465d7402e0504d10ac36635a6c0fea38 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:21.955493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:22.073517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c295-d914-43f5-bd46-d02487b442d9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:22.091642 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:22.113158 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:22.113988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:22.114203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:22.114479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:22.190648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:22.192108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:22.306313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 157/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:22.311941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:22.429306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0eb465-4453-4bc3-b1ae-367690a1790c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:22.443423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:22.466967 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:22.467836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:22.468125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:22.468421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:22.475408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:22.475408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c502162-5166-47ac-984e-20e4e315d42e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:22.477335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 160/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:22 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:22.482979 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:22.512452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:22.605565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d8fa-1022-46c1-a157-e2c802c40bb5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:22.642557 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:22.643734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:22.644243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:22.644780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:22.645850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, scope.project.id=500dd025438549d9aed4441f992c1e58, user.id=465d7402e0504d10ac36635a6c0fea38) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:22.646400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:22.648657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84745ed3-7801-491d-8d8c-6f23741a9356 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:22.655888 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 158/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:22 2022] GET /identity/v3/role_assignments?user.id=465d7402e0504d10ac36635a6c0fea38&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:23.480974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6103a4-a8cf-4211-aa99-a69f1e1dc9e7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:23.481656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6103a4-a8cf-4211-aa99-a69f1e1dc9e7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:23.481656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6103a4-a8cf-4211-aa99-a69f1e1dc9e7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:23.483040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 161/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:23.488218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:23.488695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:23.488881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:23.509005 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:23.510327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:23.597741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 159/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:23.631464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6103a4-a8cf-4211-aa99-a69f1e1dc9e7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:23.780174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6103a4-a8cf-4211-aa99-a69f1e1dc9e7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:23.815491 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:23.816298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:23.816616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:23.816899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:23.829692 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-69926202-6b7f-406c-8715-5dc3a63d208d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:23.838629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:23.839727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:23.840770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 162/321] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:23.847611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:23.860305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:23.923073 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04055-fc5c-41b5-82f0-145c272de047 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:23.923387 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04055-fc5c-41b5-82f0-145c272de047 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:23.923639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c04055-fc5c-41b5-82f0-145c272de047 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:23.925315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 163/322] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:23.931895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:23.932287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:23.932498 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:23.951195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:23.952475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:23.975479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cbce49-18e6-4b69-8281-3151dc49f083 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:23.991143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.033359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.034100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.034455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.035302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.035703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:24.036028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.038080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.047608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 164/323] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:24.048423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 160/324] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.054434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.069941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.152734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.169831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2552413-1893-4fb9-8ddd-59eb5efc649a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.187250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6277-d74c-4a77-ac27-e869153c1555 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.206508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.207390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.207679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.207953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] PATH_INFO: `/v3/users/9b83a17e264141c6a848a459b26a5ad8` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.209182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] RBAC: Authorizing `identity:get_user(user_id=9b83a17e264141c6a848a459b26a5ad8)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.211442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10145e2-feb5-4b79-8fa4-6a86d3063467 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.213131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 165/325] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/users/9b83a17e264141c6a848a459b26a5ad8 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.214309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dba8-4abf-475c-aa1c-a6e6748716ef None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.214618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dba8-4abf-475c-aa1c-a6e6748716ef None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.214896 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26dba8-4abf-475c-aa1c-a6e6748716ef None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.216133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 166/326] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:24.218825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26dba8-4abf-475c-aa1c-a6e6748716ef None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.228312 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.229160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.229444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.229721 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.240863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-b2744a08-6f03-41d7-8369-3175be132a09 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:24.249510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:24.250611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:24.251685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 161/327] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:24.253176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.253176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.253176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.267282 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.269888 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:24.271092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:24.339074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26dba8-4abf-475c-aa1c-a6e6748716ef None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.350269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 162/328] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:24.351199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.379531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.380193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.380508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.381627 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] RBAC: Adding query filter params (role.id=afa620995009427092fc6164c9cc643b, user.id=9b83a17e264141c6a848a459b26a5ad8, scope.system=all) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:24.382449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.383911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.398997 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 167/329] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/role_assignments?user.id=9b83a17e264141c6a848a459b26a5ad8&scope.system=all&role.id=afa620995009427092fc6164c9cc643b => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.400280 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.420199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.475601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9d5a4-9aab-4d4f-a602-09fc487bc94b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.520038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.520813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.521174 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.521528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.522404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:24.523347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.524897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.540240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 163/330] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.541571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.542643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.560291 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81c39f-0227-4c82-a1e0-9e33c55d74c3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.605754 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.606362 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.606652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.606940 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.608887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.609863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.611643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 168/331] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.618513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.679348 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78adc8-df1c-4376-84e7-fdc9bfce6858 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.702837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:24.716615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.717509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.717795 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.718070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] PATH_INFO: `/v3/users/111a0f7142194f949a55512c46091b05` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.719296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] RBAC: Authorizing `identity:get_user(user_id=111a0f7142194f949a55512c46091b05)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.720789 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.722496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 164/332] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:24 2022] GET /identity/v3/users/111a0f7142194f949a55512c46091b05 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:24.728859 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:24.734702 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70765253-816c-4a94-ace4-818a071712d7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.772900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.773657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.773878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.774158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] PATH_INFO: `/v3/groups` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.774854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.776313 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151420c9-6bb6-420b-80fd-7a7d66dd5c1f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.790684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 169/333] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 19:56:24 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:24.842018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81aa51-cd11-4c5f-a7d4-69a6c51f2b53 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:24.880754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:24.881538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:24.881832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:24.882853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] PATH_INFO: `/v3/system/users/111a0f7142194f949a55512c46091b05/roles/6d2b1080d0d54e43a997657c5492ad28` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:24.884537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=111a0f7142194f949a55512c46091b05, role_id=6d2b1080d0d54e43a997657c5492ad28)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:24.885960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be42cf-5a4b-4953-9ca1-963a80e6bb4e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:24.898120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 165/334] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 19:56:24 2022] PUT /identity/v3/system/users/111a0f7142194f949a55512c46091b05/roles/6d2b1080d0d54e43a997657c5492ad28 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:56:27.567712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4d52-c5f4-4a0b-86ae-8ae69605d03e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.568879 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4d52-c5f4-4a0b-86ae-8ae69605d03e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.569951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4d52-c5f4-4a0b-86ae-8ae69605d03e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.572079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 170/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:27.578713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.579706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.580212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.605685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:27.607327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:27.668439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:27.693239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 166/336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:27.719541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4d52-c5f4-4a0b-86ae-8ae69605d03e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:27.748894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:27.862134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4d52-c5f4-4a0b-86ae-8ae69605d03e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:27.907918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:27.909139 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.909653 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.910138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.923969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8a15e801-00c8-40bd-8336-5c0510c85234 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:27.933183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:27.934267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:27.935190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:27.941982 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 171/337] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:27.948957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:27.980494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb447a9-a234-460f-844a-b91f678eafed None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.981179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb447a9-a234-460f-844a-b91f678eafed None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.981686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb447a9-a234-460f-844a-b91f678eafed None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:27.983573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 172/338] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:27.998402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb820-baae-42f9-b076-9d11450ddec4 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:27.998748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb820-baae-42f9-b076-9d11450ddec4 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:27.998945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcb820-baae-42f9-b076-9d11450ddec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.026960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-4cbcb820-baae-42f9-b076-9d11450ddec4 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:28.028718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-4cbcb820-baae-42f9-b076-9d11450ddec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:28.117272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260358c7-0225-425a-9300-db7dc1f634eb None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.150321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 173/339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:28.151809 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbfc8-c349-4f83-ab89-f037bcd8323e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.152107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbfc8-c349-4f83-ab89-f037bcd8323e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.152390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bbfc8-c349-4f83-ab89-f037bcd8323e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.154051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 174/340] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:28.161632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.161632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.161632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.167454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:28.169510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.171345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.171835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.173085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:28.173823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:28.176086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:28.196148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 167/341] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:28.201864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:28.202372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:28.203233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:28.219972 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:28.264214 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:28.327125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 175/342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:56:28.328123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:28.329048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285d9e-480d-44ed-b373-352156f4c188 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.433795 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:28.434639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.434938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.435224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.476445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:28.488332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d32ee-cb2c-417d-b34d-dc7eb3329d6a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:28.488863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:28.489860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:28.490783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 168/343] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:28.492725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:28.558885 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:28.559651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.561028 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.561170 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.592797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:28.681342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:28.683109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157c8b-ec34-4c48-aeda-6a40a9afeaa0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:28.689289 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587576a2-41d2-4923-a1f2-69264de67c28 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.704560 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:28.752808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:28.754092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.754377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.755071 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.769742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:28.780308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:28.781604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:56:28.782724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:28.785857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 169/344] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:28.787083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:28.856798 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 176/345] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:28.858517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61b7c-6eec-4900-9e4a-8d21b002343a None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.859011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61b7c-6eec-4900-9e4a-8d21b002343a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.859657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61b7c-6eec-4900-9e4a-8d21b002343a None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.861611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 177/346] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:28.862513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61b7c-6eec-4900-9e4a-8d21b002343a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:28.923570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:28.942891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842b4e2-945c-4a53-9ec6-8fd321cdf9a8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.995589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:28.996277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:28.996604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:28.996890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:28.997663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:28.997900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61b7c-6eec-4900-9e4a-8d21b002343a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:28.998021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:28.999369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.009730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 170/347] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.010422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.048341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.050168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.050168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.050168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.051151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:29.051469 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.052036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.054616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898921b-adb5-441c-a18b-9032caf0ccc3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.064931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 178/348] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.067143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9a8-7b6c-47ec-ac4e-6f2bb5c2666d None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.067593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9a8-7b6c-47ec-ac4e-6f2bb5c2666d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.068500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9a8-7b6c-47ec-ac4e-6f2bb5c2666d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.090570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-fab0f9a8-7b6c-47ec-ac4e-6f2bb5c2666d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:29.091776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-fab0f9a8-7b6c-47ec-ac4e-6f2bb5c2666d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:29.150975 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50eed3-d184-48eb-98cf-cdb4aeda87e0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.189371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 179/349] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:29.190778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20586ee-e692-4aa1-adf3-23baae217acd None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.191072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20586ee-e692-4aa1-adf3-23baae217acd None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.191705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20586ee-e692-4aa1-adf3-23baae217acd None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.197198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 180/350] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:29.197801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20586ee-e692-4aa1-adf3-23baae217acd None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.210819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.212151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.212737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.213746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.215756 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.217581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.219809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 171/351] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.220814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.252381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.348930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.353163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20586ee-e692-4aa1-adf3-23baae217acd None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.355817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd97af-3519-47fb-a227-0a65baa3e1fd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.399620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.400298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.400709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.400895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.414301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.415257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.415546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.415823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] PATH_INFO: `/v3/users/111a0f7142194f949a55512c46091b05` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.417108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] RBAC: Authorizing `identity:get_user(user_id=111a0f7142194f949a55512c46091b05)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.418582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e39e2-9852-4006-92a0-8bdfa2d9113c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.420606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 172/352] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/users/111a0f7142194f949a55512c46091b05 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.421900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.422191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.422725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.442163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:29.443403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:29.513131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.517216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.542205 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 173/353] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:29.543171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.573926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.654993 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 181/354] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:29.657325 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.683221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.691919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bb2d9-a1a5-4cd6-91b7-3705bd2a34cc None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.739898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.740779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.741490 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.741811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.756353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:29.766285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:29.767359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:29.768271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 174/355] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:29.769127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.792236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756224-c28b-4bca-8df0-8038a14bc22c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.840327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.841239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.841528 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.841806 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.845861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:29.850344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.852088 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.854110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 182/356] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.855074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:29.917566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c04cb-ba67-468f-9678-7a59c82d4d09 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.972480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:29.973165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:29.973450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:29.973725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:29.974567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] RBAC: Adding query filter params (role.id=6d2b1080d0d54e43a997657c5492ad28, user.id=111a0f7142194f949a55512c46091b05, scope.system=all) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:29.974882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:29.977377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:29.982962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3110e37b-b471-4ede-9314-01b1b3f2f774 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:29.998295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 175/357] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/role_assignments?user.id=111a0f7142194f949a55512c46091b05&scope.system=all&role.id=6d2b1080d0d54e43a997657c5492ad28 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:29.998295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.018706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.042857 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.043697 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.043987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.044267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.061701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-703b24d0-7935-41f3-9250-ac2f64b5f03b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:30.075694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:30.076856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:30.077797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:30.078494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 183/358] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:30.079621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205605d1-40e6-4c1e-92f0-c35cfb25fe24 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.079918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205605d1-40e6-4c1e-92f0-c35cfb25fe24 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.080203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205605d1-40e6-4c1e-92f0-c35cfb25fe24 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.083477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 184/359] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:30.083477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205605d1-40e6-4c1e-92f0-c35cfb25fe24 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.153668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a7f6-3413-4d69-889a-60ee804fa027 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.173414 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.202964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.204241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.204730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.205006 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.206237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.207707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.209816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 176/360] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:29 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:30.210851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.239835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205605d1-40e6-4c1e-92f0-c35cfb25fe24 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.283652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.284720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.285236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.285740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.286718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:30.287030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.289119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.299690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 185/361] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:30.300346 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.322258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.349864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fed16-3c2a-42c2-8dda-6cf6898f860f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.395953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.396670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.399935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.399935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.399935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=afa620995009427092fc6164c9cc643b, scope.project.id=1557b854d9814d43a2958ac6f6f85e48) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:30.399935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.400538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878472e-99a5-45e1-ac2d-98837bb153ae None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.407096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 177/362] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=afa620995009427092fc6164c9cc643b => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:30.408598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.408903 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.409194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.430403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:30.431967 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:30.443851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821afc1-ae3e-4dc2-9ae5-f62987d1db2f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.479064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.495413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.496119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.496494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.496716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.497870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:30.498185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.500330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.516474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 186/363] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:30.516474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.549410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 178/364] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:30.550653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.663993 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.671166 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e3d0f-eee0-4ee9-b174-96635788e4a5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.702957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d4ca0-3f71-47f1-a32c-9c94eeab6bf4 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.726533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.727454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.728229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.728597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.729958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.731418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.735773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 187/365] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:30.735773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:30.764716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.765442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.767654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.767654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:30.833858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:30.880083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:30.882575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:30.891298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb6fb9-c38f-4dbc-adeb-75b07d2e1cb7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:30.931930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:30.932699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:30.932988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:30.933270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.004584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.030324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.031853 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.044492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 179/366] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:31.047048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.101320 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.172710 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 188/367] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:31.173882 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.192715 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc5785-5882-428a-bba9-4d4d55f4ae1a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.241488 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.242298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.242580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.242853 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.253616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:31.262669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:31.263730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 180/368] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:31.265442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.301721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e334cf-5e67-49a8-98e1-5925a2d1c005 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.304970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.341561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.342345 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.342637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.342923 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.343762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=1557b854d9814d43a2958ac6f6f85e48) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:31.344072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.345574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.360516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 189/369] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=dd3a79631b06401fab3c7670acb9024d => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.360516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.375629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.424051 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fbd6f-4661-4615-b391-824aa8648ecf None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.484269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.484525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d9c60-6bed-4d49-be57-17853b44f7af None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.485235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.485524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.485804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.488222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.489742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.491910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 181/370] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.492589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.524944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.525990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.526273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.526548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.528011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.529618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.531916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 190/371] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.532523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.648459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9589103-510c-4804-b73a-be6c143fa71b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.651311 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.652068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.659583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482ba38-81e0-4b7c-aa80-df542bc89619 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.692284 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.693007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.693357 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.693635 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.695472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:31.695791 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.697234 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.710763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.711624 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 182/372] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.712578 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.713481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.714009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.714500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.716042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.717861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.720063 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 191/373] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.721115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.831913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2fc55-23a1-4795-82cf-581e96b21f5a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.842427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba65771-c543-435a-81c0-7561b0b8cf78 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:31.871535 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.872675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.872818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.873073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.874274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.875829 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.877787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 183/374] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:31.883899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:31.886185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:31.887231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:31.887842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:31.888336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:31.914611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.924128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:31.973869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:31.976354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:31.995328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f50e1d-5c64-4241-9da2-5f1fcc0cebb8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:32.032937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:32.033461 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.033743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.034022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.034857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=afa620995009427092fc6164c9cc643b, scope.project.id=500dd025438549d9aed4441f992c1e58) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:32.035172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:32.036678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:32.042895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 184/375] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=afa620995009427092fc6164c9cc643b => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:32.049420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:32.096902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 192/376] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:32.103577 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:32.167919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:32.225202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a684c0-4cf0-4a0f-9421-e4dc1342d071 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:32.243974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:32.256591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4008e9-9823-449b-af8a-efa3bffc70cd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:32.269606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:32.270907 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.271442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.271948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] PATH_INFO: `/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.273792 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a0700fd9a2e34b6cbec11fb0d6875eef)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:32.275559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:32.278109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 193/377] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:32 2022] GET /identity/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:32.283930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:32.310634 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:32.311143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.311427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.311705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.312582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=500dd025438549d9aed4441f992c1e58) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:32.312941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:32.314536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:32.321261 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 185/378] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:32 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=dd3a79631b06401fab3c7670acb9024d => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:32.420184 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a751d5-956d-417d-bcd0-35fbf47bd8f0 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:32.444956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:32.462045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:32.463348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.463863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.464364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] PATH_INFO: `/v3/groups/92b6c778203c4280abd8ae00e6176a22` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.470712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] RBAC: Authorizing `identity:get_group(group_id=92b6c778203c4280abd8ae00e6176a22)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:32.472672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac319866-404e-4f04-94df-a4fc753a9fcf None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:32.475182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 194/379] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:32 2022] GET /identity/v3/groups/92b6c778203c4280abd8ae00e6176a22 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:32.485584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:32.622753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c8b49-1b77-4d64-a393-c8bce7579d37 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:32.670691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:32.671286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.671570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.671846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.673000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] RBAC: Adding query filter params (group.id=92b6c778203c4280abd8ae00e6176a22, role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=a0700fd9a2e34b6cbec11fb0d6875eef) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:32.673344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:32.674757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013154a3-c349-4be2-813d-323fb010dcea None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:32.682365 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 186/380] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:32 2022] GET /identity/v3/role_assignments?group.id=92b6c778203c4280abd8ae00e6176a22&scope.project.id=a0700fd9a2e34b6cbec11fb0d6875eef&role.id=771c065274f04f49bcd4641d79a0e490 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:32.862288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66912b6-9117-44c1-a86d-c74e7e335de1 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.862818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66912b6-9117-44c1-a86d-c74e7e335de1 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.862818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66912b6-9117-44c1-a86d-c74e7e335de1 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.864610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 195/381] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:32.877463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:32.877997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:32.877997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:32.905599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:32.906849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:32.930622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:32.985759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 187/382] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:33.021217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66912b6-9117-44c1-a86d-c74e7e335de1 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:33.165472 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66912b6-9117-44c1-a86d-c74e7e335de1 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:33.216204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:33.216958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:33.217238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:33.217507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:33.233424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:33.241990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:33.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:33.243987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 196/383] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:33.251254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:33.397399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679ce0a-7900-4528-9b93-0a885ff8f8ef None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:33.436455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:33.437133 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:33.437415 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:33.437690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:33.438443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:33.438752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:33.440120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:33.450001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 188/384] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:33.455674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:33.534677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:33.615155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857cf2-30c8-4662-9507-807062556359 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:33.665572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:33.666499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:33.666780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:33.667052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:33.673721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:33.675232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:33.677401 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 197/385] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:33 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:33.683604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:33.729639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:33.861947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef561-daf5-4ae9-970a-950d7b3c3892 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:33.917875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:33.919556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:33.920060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:33.920620 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:33.922062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:33.924025 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbea849-beea-4e91-9ac2-d6d716423ab1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:33.927011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 189/386] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:33 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:33.935501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:34.101571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c488330-f9c1-4d81-9e6e-3e1fb159b4c6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:34.131454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:34.185657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:34.187213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.187619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.188021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/afa620995009427092fc6164c9cc643b` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.194574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, group_id=70dbde63036a4c8aa1983a9fb606f7e9, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:34.196727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96397881-6569-4034-bc49-b3eefc77bf4e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:34.221087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 198/387] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:56:33 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:34.265562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b167f83-74d7-4157-92b3-3cf36a90346b None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.266174 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b167f83-74d7-4157-92b3-3cf36a90346b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.266639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b167f83-74d7-4157-92b3-3cf36a90346b None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.268659 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 190/388] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:34.274947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.275574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.276066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.295657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:34.297893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:34.395761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 199/389] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:34.419866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b167f83-74d7-4157-92b3-3cf36a90346b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:34.550994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:34.577121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b167f83-74d7-4157-92b3-3cf36a90346b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:34.632680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:34.633515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.633804 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.634083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.659202 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:34.675389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:34.676553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:34.677509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 191/390] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:34.684843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:34.826211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407bc13-c281-4af5-9b34-d36e627954b7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:34.828280 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:34.865366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:34.866158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.866461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.866795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.867573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:34.867887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:34.869542 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d850f9a-9095-4973-a499-a7ed5bd33fc5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:34.888070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 200/391] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:34.893827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:34.982650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079da426-67c5-45de-8c38-42c5c03d6808 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.983162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079da426-67c5-45de-8c38-42c5c03d6808 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.983162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079da426-67c5-45de-8c38-42c5c03d6808 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:34.984812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 201/392] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:34.990209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:34.990510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:34.990789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.018494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:35.019749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:35.086997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73383d49-2b7c-42e9-8efd-edd440d8a48e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.142133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 202/393] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:35.166173 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.181278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.182179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.182752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.183034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.184463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:35.185919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:35.187573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 192/394] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:34 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:35.193400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.217555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:35.304979 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f6c7b-ebc5-4adc-9ca2-5259c39001b4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.343658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.344687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.344972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.345249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.346450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:35.348208 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eb833-0808-4fc0-8d49-657a260139aa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:35.350385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 193/395] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:35.357733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d671d-b839-470d-ae17-6b07e3cf2f75 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.358011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d671d-b839-470d-ae17-6b07e3cf2f75 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.358405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d671d-b839-470d-ae17-6b07e3cf2f75 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.363968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 194/396] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:35.363968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d671d-b839-470d-ae17-6b07e3cf2f75 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.367122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:35.375985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a0f2a-3465-438d-a8f8-f90f9514a967 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.423930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.425347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.425889 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.426778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.443282 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-632753ce-e049-4b9f-93b0-135f7148848a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:35.455561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:35.456944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:35.458062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:35.466163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 203/397] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:35.466163 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.466428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.466513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.475951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:35.493679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:35.496133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:35.504990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d671d-b839-470d-ae17-6b07e3cf2f75 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.549096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.550161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.550505 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.550851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.553388 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] RBAC: Authorizing `identity:create_grant(project_id=1557b854d9814d43a2958ac6f6f85e48, group_id=70dbde63036a4c8aa1983a9fb606f7e9, role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:35.555807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:35.571121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 195/398] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:56:35 2022] PUT /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/dd3a79631b06401fab3c7670acb9024d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:56:35.577335 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.592478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 204/399] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:35.617351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.718004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:35.738540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a2264-89d4-4f78-9f8b-aa20892d1763 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.742660 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edaaa3-969d-456e-8d3c-27a9d823af7d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.781531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.782731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.782920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:35.783587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.783705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.783875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.784156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.784574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.784982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] RBAC: Adding query filter params (name=member) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:35.785294 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:35.786768 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523fb2e-7a76-44b3-9d17-07fdcdd26bbb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:35.802008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 196/400] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:35.802008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7aa5c-d030-45c5-9b89-cef7819b95a9 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.802008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7aa5c-d030-45c5-9b89-cef7819b95a9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.802008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7aa5c-d030-45c5-9b89-cef7819b95a9 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.804458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:35.804898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 197/401] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:35.806060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7aa5c-d030-45c5-9b89-cef7819b95a9 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:35.812457 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-d4fb0eec-54a7-4efa-bd55-b9496b2d33b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:35.816020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:35.817255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:35.818196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:35.827160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 205/402] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:35.829200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:35.829791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:35.830276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:35.856109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:35.857822 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:35.961135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7aa5c-d030-45c5-9b89-cef7819b95a9 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:35.967559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 206/403] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:35.968831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.001914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.002831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.003193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.003501 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.004769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.013299 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.020425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 198/404] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.020425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.076947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.107804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61771df7-3d29-41d2-ad2e-b6e7bf68b15e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.123832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.156968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.157681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.157974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.158255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.159017 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:36.159426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.160885 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.169714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a8cd-ff3f-4906-9eec-7440e47509c8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.179579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 207/405] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.180667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.211871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.213119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.213668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.214217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.226937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:36.235338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:36.236433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:36.237386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:36.239916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 199/406] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:36.240963 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.308205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe2d19-ad15-4551-85d1-fd487c071bac None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.345643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.346504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.346789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.347067 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.348267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.349797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.356863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 208/407] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.356863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.357131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.369116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae6a7a-669d-47ef-9431-05d7780e56a4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.401846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.408171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.410162 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.410708 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.411204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.412735 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.414973 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.417239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 200/408] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.418238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.467749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a227a3-ba82-4d06-be98-c5d3a4eadb9a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.507337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.508029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.508314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.508626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.509567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:36.509758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.511299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.523980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 209/409] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.532434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.549447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8f1f-f644-4f70-b9dd-7ee38b66fdc1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.591084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.591991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.592278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.592599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/afa620995009427092fc6164c9cc643b` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.594843 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, group_id=70dbde63036a4c8aa1983a9fb606f7e9, role_id=afa620995009427092fc6164c9cc643b)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.596483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.601462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.616969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 201/410] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:56:36 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/afa620995009427092fc6164c9cc643b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:36.618803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.675003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173578f-1919-4425-aba4-2e98728add9d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.701292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:36.757881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6664e-ff14-45e9-b89e-29b2e433e0fa None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:36.798376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.799235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.799530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.799808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.804485 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:36.805227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.805989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:36.806481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:36.806711 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.807137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] PATH_INFO: `/v3/groups/92b6c778203c4280abd8ae00e6176a22` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:36.808693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 210/411] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.809130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] RBAC: Authorizing `identity:get_group(group_id=92b6c778203c4280abd8ae00e6176a22)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:36.815710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:36.818155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 202/412] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/groups/92b6c778203c4280abd8ae00e6176a22 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:36.822093 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.824799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:36.984257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436118f-f0be-4aee-bb4a-c4d730f2ee18 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.027597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:37.038179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.039689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.040182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.040703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] PATH_INFO: `/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.042420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] RBAC: Authorizing `identity:get_project(project_id=a0700fd9a2e34b6cbec11fb0d6875eef)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:37.044213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:37.047089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 203/413] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:36 2022] GET /identity/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:37.049494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ee20c-8f91-4a0a-bbef-649a0097871e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.054369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:37.089649 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.090492 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.090778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.091058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.093384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] RBAC: Authorizing `identity:create_grant(project_id=500dd025438549d9aed4441f992c1e58, group_id=70dbde63036a4c8aa1983a9fb606f7e9, role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:37.094864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4df86-5c69-45a6-9ea8-b909f5d79eea None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:37.097322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:37.112751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 211/414] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:56:36 2022] PUT /identity/v3/projects/500dd025438549d9aed4441f992c1e58/groups/70dbde63036a4c8aa1983a9fb606f7e9/roles/dd3a79631b06401fab3c7670acb9024d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:37.114102 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b6563-579a-4d8e-83f3-c42bd7a8be33 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.114397 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b6563-579a-4d8e-83f3-c42bd7a8be33 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.114679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b6563-579a-4d8e-83f3-c42bd7a8be33 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.116551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 212/415] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:37.129032 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.129828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.129828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.169867 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:37.171020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:37.191750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c8304-b310-4e3f-a8e6-b902cb7de31f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.243552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.244902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.245407 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.245916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] PATH_INFO: `/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef/groups/92b6c778203c4280abd8ae00e6176a22/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.248375 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] RBAC: Authorizing `identity:create_grant(project_id=a0700fd9a2e34b6cbec11fb0d6875eef, group_id=92b6c778203c4280abd8ae00e6176a22, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:37.250209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:37.277774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 204/416] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 19:56:37 2022] PUT /identity/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef/groups/92b6c778203c4280abd8ae00e6176a22/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:56:37.317669 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 213/417] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:37.354170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:37.432669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:37.493064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043abdd-7610-4dfa-a99a-9e251060678d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.546095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.546927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.547213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.547493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] PATH_INFO: `/v3/roles/member` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.560106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:37.568062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:37.569343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:37.570367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 205/418] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:37.577528 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:37.726030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91825c-1cef-4265-808c-217b1c940bf9 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.758512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:37.776771 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.777835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.778357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.778852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:37.779843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:37.780379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:37.782164 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6933c-5793-43fd-a646-910ed34d509f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:37.797312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 214/419] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:37.803358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:37.934737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7563784-3969-4530-850c-cb8066a2417f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:37.955171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:37.975259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:37.976617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:37.976906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:37.977240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.059871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:38.061293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:38.145406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcb70-b49d-4de8-b931-d33074a8c11b None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.146057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcb70-b49d-4de8-b931-d33074a8c11b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.146057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fcb70-b49d-4de8-b931-d33074a8c11b None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.147850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 215/420] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:38.156958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.157324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.157520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.170219 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 206/421] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:38.175695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:38.201426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:38.202701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:38.216785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:38.354721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 216/422] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:38.366629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34886e-3e48-4219-9975-af14be924d0b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:38.400736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:38.419805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:38.420738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.421021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.421297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.422997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:38.424515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:38.433241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 207/423] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:38 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:38.437277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:38.580268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c8f2e-faf8-4a03-81f8-01076d70c012 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:38.597690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:38.624621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:38.634849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:38.635688 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.635979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.636266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.643912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4526d-f3a6-4f79-8a14-2987db1043b0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:38.658394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:38.671107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:38.672258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:38.673488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:38.680947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 217/424] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:38.688601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:38.693835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:38.695252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.695776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.696264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.697746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:38.699421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:38.701793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 208/425] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:38 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:38.707475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:38.872823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204075f7-8651-4473-8636-cc74e3c93b65 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:38.918419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da1951-7f86-47c5-8ab3-aeaea6b10173 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:38.920694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:38.921746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:38.922231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:38.922973 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:38.924025 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=afa620995009427092fc6164c9cc643b, scope.project.id=1557b854d9814d43a2958ac6f6f85e48) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:38.924583 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:38.927564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:38.938620 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 209/426] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:38 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=afa620995009427092fc6164c9cc643b => generated 611 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:39.023216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:39.024063 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.024352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.024652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.025452 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:39.025764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:39.027160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:39.047232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 218/427] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:39.053312 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.116954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:39.205204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b05a48-a127-4e87-a711-e469b1e48d88 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:39.243844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:39.244810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.245302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.245706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.358324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:39.360971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:39.382137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:39.494418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 210/428] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:39.499563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.575758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:39.644502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b891-f267-4546-850a-4b7d92de5106 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:39.692581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:39.693473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.693759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.694039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] PATH_INFO: `/v3/projects/1557b854d9814d43a2958ac6f6f85e48` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.696433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] RBAC: Authorizing `identity:get_project(project_id=1557b854d9814d43a2958ac6f6f85e48)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:39.697933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f74292-e775-460b-84ee-47bb66887f37 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:39.699990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 219/429] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/projects/1557b854d9814d43a2958ac6f6f85e48 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:39.707358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.750189 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897173-694e-44e7-a440-87b4b6dabdff None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.750404 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897173-694e-44e7-a440-87b4b6dabdff None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.750675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897173-694e-44e7-a440-87b4b6dabdff None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.752499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 220/430] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:39.758514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.758811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.759099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.779757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:39.780997 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:39.845918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dded579-81b5-4152-a322-bf9b8fa7cc33 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:39.867630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 221/431] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:39.883578 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:39.895568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:39.896472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:39.896766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:39.897045 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:39.898580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:39.900383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:39.901268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.902353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 211/432] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:39.909332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:39.956278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.066819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa360b8-c542-407e-aac4-b12564e6ccdb None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.082148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7237c-cb41-4283-a47d-3a2f0231af86 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.114112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.115217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.115511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.115793 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.121599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.122262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.122551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.122832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.123696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=1557b854d9814d43a2958ac6f6f85e48) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:40.124000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:40.125550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fe3d2-e2e8-44c2-a4cc-bafcea0e5b06 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:40.130980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:40.132884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 212/433] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=1557b854d9814d43a2958ac6f6f85e48&role.id=dd3a79631b06401fab3c7670acb9024d => generated 611 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:40.134426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600df818-dc6d-478c-a1f9-6f2885636f2d None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.134728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600df818-dc6d-478c-a1f9-6f2885636f2d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.135013 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600df818-dc6d-478c-a1f9-6f2885636f2d None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.136710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 213/434] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:40.141591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:40.142873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:56:40.143887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:40.144652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4febf-5423-4356-9730-043c5a5946c7 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.144652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4febf-5423-4356-9730-043c5a5946c7 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.144652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4febf-5423-4356-9730-043c5a5946c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.146703 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 222/435] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:40.154036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.165060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-d7e4febf-5423-4356-9730-043c5a5946c7 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:40.166384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-d7e4febf-5423-4356-9730-043c5a5946c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:40.218986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.250231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 214/436] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:40.253505 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b24b4-7f0b-49e0-8199-34cbed0404df None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.254354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b24b4-7f0b-49e0-8199-34cbed0404df None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.254801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b24b4-7f0b-49e0-8199-34cbed0404df None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.257590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 215/437] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:40.266468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.266748 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.267024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.281032 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.290519 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:40.291902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b907fa-8996-406a-9fe9-08c461ac2dc2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.292370 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:40.336339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.336992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.337281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.337568 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.338359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] RBAC: Adding query filter params (name=member) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:40.338690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:40.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:40.351215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 223/438] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:40.351215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.384892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 216/439] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:40.386014 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.491742 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a85764-209e-4bee-9cb0-7cf1ba31e179 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.520171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afda4b-f573-4e64-bb35-285a64438f20 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.533450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.533968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.534264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.534578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.534878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.551661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:40.560129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:40.561209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:40.562118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 224/440] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:40.562879 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.567665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.568621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.569257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.570268 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.585294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.648310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:40.649992 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:40.675496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d077b1c-2493-448b-9eca-3e70b8647fc4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.712096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.712951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.713231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.713505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.724246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:40.733361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:40.734523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 225/441] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:40.735595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.757916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 217/442] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:40.758957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.769372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.829730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:40.863558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2517662-d667-424e-a8e1-7a44cfa8049b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.864740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01982032-0ff4-41cc-b8fd-5fde962315e7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:40.901646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.902257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.902544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.902645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:40.902820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.903242 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:40.903530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:40.903641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:40.903810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:40.903906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:40.904590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:40.904905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:40.905315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:40.906341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:40.916017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 218/443] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:40.916917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.920594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 226/444] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:40.922854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:40.989290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.024130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e64fb-af31-4526-872b-70dfdc958a22 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.027302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6073266-d1c3-4498-91bd-2e0f43d05f86 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.058957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.059819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.060104 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.060414 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.062357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.063021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.063307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.063584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.070942 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.072478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.074308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 219/445] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:41.075178 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.077835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.140434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.141944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.184946 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75f5d-2484-4197-b14e-34571f7d01f8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.218751 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.219416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.219707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.219985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.230257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.253228 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 227/446] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:41.254083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.299486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.300612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.302387 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.362040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c1b29-074a-423f-a338-23028715c6c4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.398293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.399160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.399440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.399722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.400916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.402379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.403993 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 228/447] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:41.404769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.410423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 220/448] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:41.410423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.451627 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.514896 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cb190-cb44-41cb-b1d5-f7fb56ffe27a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.520960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd75d4-d93f-44b1-b862-399ffe0dacbb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.526254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.557020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.557991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.558282 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.558560 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] PATH_INFO: `/v3/projects/500dd025438549d9aed4441f992c1e58` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.559135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.559795 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.560039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] RBAC: Authorizing `identity:get_project(project_id=500dd025438549d9aed4441f992c1e58)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.560148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.560349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.561223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=afa620995009427092fc6164c9cc643b, scope.project.id=500dd025438549d9aed4441f992c1e58) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:41.561538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.561633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.562969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.563277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 229/449] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/projects/500dd025438549d9aed4441f992c1e58 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:41.564042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.569149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 221/450] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=afa620995009427092fc6164c9cc643b => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:41.569951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.705736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5a06d-68d1-4f77-a41b-8c8d7341d27d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.716357 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90b1c4-e154-48f9-9b6b-223e9e3276b4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.718479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.748183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.749182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.749566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.749566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] PATH_INFO: `/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.751094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] RBAC: Authorizing `identity:get_project(project_id=a0700fd9a2e34b6cbec11fb0d6875eef)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.752641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.754603 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 230/451] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/projects/a0700fd9a2e34b6cbec11fb0d6875eef => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:41.760457 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.762282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.763455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.764016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.764298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] PATH_INFO: `/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.769197 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] RBAC: Authorizing `identity:get_group(group_id=70dbde63036a4c8aa1983a9fb606f7e9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.770754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.772668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 222/452] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/groups/70dbde63036a4c8aa1983a9fb606f7e9 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:41.778523 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.814668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:41.884682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e7471-c8b9-45d2-a594-f1f123b2899f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.908170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cab669-259f-4b8e-a6a7-785aeef37d8a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:41.933515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.934707 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.935156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.935617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] PATH_INFO: `/v3/groups/92b6c778203c4280abd8ae00e6176a22` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.937175 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] RBAC: Authorizing `identity:get_group(group_id=92b6c778203c4280abd8ae00e6176a22)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.938660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.940619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 231/453] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/groups/92b6c778203c4280abd8ae00e6176a22 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:41.948634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:41.954459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:41.955290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:41.955584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:41.955964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:41.956816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] RBAC: Adding query filter params (group.id=70dbde63036a4c8aa1983a9fb606f7e9, role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=500dd025438549d9aed4441f992c1e58) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:41.957140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:41.958649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c0d86b-b73e-4f84-b12a-17ed335a1102 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:41.965591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 223/454] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/role_assignments?group.id=70dbde63036a4c8aa1983a9fb606f7e9&scope.project.id=500dd025438549d9aed4441f992c1e58&role.id=dd3a79631b06401fab3c7670acb9024d => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:42.077638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee518ba7-6969-4a20-a53b-032b4f166d0d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:42.093634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:42.118987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:42.119660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:42.119948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:42.120227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:42.121119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] RBAC: Adding query filter params (group.id=92b6c778203c4280abd8ae00e6176a22, role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=a0700fd9a2e34b6cbec11fb0d6875eef) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:42.121440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:42.122913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d39df-6469-4d72-a875-ebd891778586 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:42.129608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 232/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 19:56:41 2022] GET /identity/v3/role_assignments?group.id=92b6c778203c4280abd8ae00e6176a22&scope.project.id=a0700fd9a2e34b6cbec11fb0d6875eef&role.id=771c065274f04f49bcd4641d79a0e490 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:44.918376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14de89-4942-4fe2-a7e5-22af7a46eb12 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:44.919448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14de89-4942-4fe2-a7e5-22af7a46eb12 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:44.920037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14de89-4942-4fe2-a7e5-22af7a46eb12 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:44.921677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 224/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:44.930599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:44.931632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:44.932142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:44.957425 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:44.959023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:45.054944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 233/457] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:45.101074 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14de89-4942-4fe2-a7e5-22af7a46eb12 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.163064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:45.249364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14de89-4942-4fe2-a7e5-22af7a46eb12 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:45.297149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:45.298237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.298527 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.298796 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.316741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:45.318790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-61304fef-5873-4384-813f-5c6ef14bbb2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:45.350746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:45.351816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:45.352860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:45.352860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:45.352860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:45.352860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:45.354950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 225/458] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:45.360843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.375952 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828cd67-f998-47fe-bc18-538ac0168424 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.376308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828cd67-f998-47fe-bc18-538ac0168424 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.376559 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828cd67-f998-47fe-bc18-538ac0168424 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.379335 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 226/459] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:45.385709 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858444c-f1c2-4980-a762-9e83336ff28d None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.385994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858444c-f1c2-4980-a762-9e83336ff28d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.386274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858444c-f1c2-4980-a762-9e83336ff28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.418429 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2858444c-f1c2-4980-a762-9e83336ff28d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:45.419635 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2858444c-f1c2-4980-a762-9e83336ff28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:45.556652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 227/460] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:45.558138 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4f0e5-ad95-42a6-9db7-413a9b780742 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.558691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4f0e5-ad95-42a6-9db7-413a9b780742 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.559223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4f0e5-ad95-42a6-9db7-413a9b780742 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.565579 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 228/461] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:45.567290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.567743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.568171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.578298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8be577-374b-45b9-af69-cb6d092224cb None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:45.596093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:45.596690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:45.597937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:45.612796 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:45.636927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:45.638355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.638955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.639517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.640663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:45.641144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:45.643766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:45.661595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 234/462] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:45.662908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.690342 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 229/463] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:45.691562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.798779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baa3e3-4b0c-4d02-b471-d17b8ff6ac05 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:45.803070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff7d1a-2ae7-4cd7-8da7-32c61ecb371f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:45.839863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:45.840693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.840976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.841253 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.852436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:45.852632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:45.853323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.853619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.853898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.854014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:45.854611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:45.856179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-5620aff4-4a9e-47e8-8db4-4524b6d6cd0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:45.875219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:45.876318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 235/464] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3ce-de20-44db-96e1-62cf57d732aa None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3ce-de20-44db-96e1-62cf57d732aa None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:45.883124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ae3ce-de20-44db-96e1-62cf57d732aa None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:45.884900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 236/465] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:45.885871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae3ce-de20-44db-96e1-62cf57d732aa None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.888960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 230/466] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:45 2022] POST /identity/v3/users => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:45.890064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:45.893077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:45.916342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.035041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ae3ce-de20-44db-96e1-62cf57d732aa None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.063106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae47b1f-9e5b-45d5-8db8-ca136b7758fa None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.087138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.088278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.088817 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.089295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.102471 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.104064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.107568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.108243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.108565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.108842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.109570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:46.109881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.111264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-adff3959-c81c-4915-8219-e03ea04a6e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:46.125494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:46.127176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 237/468] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:46.129983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.131283 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 231/468] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:46.133554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.133763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.133763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.158376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:46.159598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:46.227824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.255579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.262687 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 238/469] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:46.263588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.286487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab477dc7-e032-4ffd-85d7-e3899f4da8bc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.338354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.339475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.340571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.340908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.341677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.343188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.377141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 232/470] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:56:46.378450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.411979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a56e0-6351-43d6-9f63-f752ac42d217 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.453612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.454010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.454271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.454549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.455318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:46.455656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.457155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.471011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 239/471] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:46.477848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.520211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c00acd-2cb3-477f-8218-17dfb5a0158d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.545711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.568302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.569694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.570201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.570740 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.571981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.583402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.586146 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-da4aeddd-7abb-4511-8cf1-e1714257b51a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:46.606915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:46.607939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:46.608945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:46.608945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:46.608945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:46.608945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:46.611348 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 233/472] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:46.613001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb54a70-c3fd-4c3d-93cd-709c81a239c7 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.613553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb54a70-c3fd-4c3d-93cd-709c81a239c7 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.614046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb54a70-c3fd-4c3d-93cd-709c81a239c7 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.615665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 234/473] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:46.618702 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb54a70-c3fd-4c3d-93cd-709c81a239c7 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.631445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bf26b-5c57-4d41-a2b1-1047ad67748e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.672274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.673059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.673374 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.673668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.674408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.676309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ef148-72f9-4825-b1d3-9d08d6f79131 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.707715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 240/474] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:56:46.709422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.709761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.712744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.733069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:46.734320 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:46.770860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb54a70-c3fd-4c3d-93cd-709c81a239c7 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:46.826045 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:46.827160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:46.827665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:46.827781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 241/475] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:46.828286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:46.829287 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:46.829840 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:46.831495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:46.847591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 235/476] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:46.848839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.854733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:46.904984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.911394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:46.987552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339a4be-59a4-4167-9ba2-eb07ee819000 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:47.017092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d46c7-0b13-4c7f-b1c5-505715e690b5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:47.031898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:47.033002 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:47.033540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:47.034031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:47.035176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:47.037348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:47.046301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.sql.core [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:56:47.046301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:56:47.046301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [parameters: {'id': '80ad836220d04b728094cdaceb43d826', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:56:47.046301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150737) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:56:47.066400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:47.067148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:47.067534 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:47.067828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:47.068609 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:47.070965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application conn.query(q) May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result.read() May 19 19:56:47.071459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return ref.to_dict() May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:56:47.072477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application next(self.gen) May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application yield resource May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application next(self.gen) May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise self.value May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application session.commit() May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:56:47.073602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.session.flush() May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application compat.raise_( May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:56:47.074612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:56:47.076633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application conn.query(q) May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result.read() May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:56:47.078823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [parameters: {'id': '80ad836220d04b728094cdaceb43d826', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:47.081043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:56:47.083285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.087571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 242/477] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/roles => generated 144 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:47.087571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:47.119556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 236/478] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:46 2022] POST /identity/v3/users => generated 249 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:56:47.222107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c9753-5ff3-44ad-8017-5a08664ffd6e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:47.234972 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:47.266827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:47.267675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:47.268013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:47.268295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:47.286055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-0eb48915-d5c0-4269-b1c8-ca59b984d0a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:47.295221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:47.296360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:56:47.297305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:47.300523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 243/479] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:56:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:47.304046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:47.424723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:47.443598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadcdebd-9d50-4a2b-a2c9-7ef3acde428a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:47.495443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:47.496053 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:47.496336 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:47.496634 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:47.497392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:47.497700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:47.499085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132ea0a-d13a-4245-bcd5-edc153e78acc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:47.526121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 237/480] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:48.474785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e81c8-6350-4c7c-b6ad-00277a6daa7a None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:48.476499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e81c8-6350-4c7c-b6ad-00277a6daa7a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:48.477035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e81c8-6350-4c7c-b6ad-00277a6daa7a None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:48.478920 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 244/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:48.484741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:48.485621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:48.486143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:48.504238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:48.505863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:48.586097 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 238/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:48.610262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e81c8-6350-4c7c-b6ad-00277a6daa7a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:48.756315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e81c8-6350-4c7c-b6ad-00277a6daa7a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:48.794179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:48.795402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:48.795970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:48.796482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:48.809153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:48.821705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:48.822780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:48.823712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:48.826821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 245/483] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:48.833153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:48.919834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:48.953137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e1936-a55f-4748-8861-e64aad7b605b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.019011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.019619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.019908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.020293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.021097 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:49.021430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:49.023520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:49.036734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 239/484] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:49.043762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:49.099923 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:49.214533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e466091-9ffd-48aa-9cde-327ad2cc87eb None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.265376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.266185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.266471 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.266748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.284763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:49.286192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-5e738128-64f8-41f7-a3a6-f9f5f0af8808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:49.309864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:49.310919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:49.311901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:49.311901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:49.311901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:49.311901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:49.313911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 246/485] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:49.318663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:49.376073 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adef0-c006-4e33-a331-13211caebb3f None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.376447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adef0-c006-4e33-a331-13211caebb3f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.376655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adef0-c006-4e33-a331-13211caebb3f None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.378340 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 247/486] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:49.394620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.394832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.395052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.436368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:49.437606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:49.448766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767f27a-8a14-451f-8375-481371dc460f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.489895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.491965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.492499 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.492997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.494427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:49.494973 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:49.496646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:49.516431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:49.517293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 240/487] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:49.519295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:49.639181 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4de07-0270-47d1-8d2d-c869f4f437bb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.697963 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.699166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.699670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.700166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.708869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:49.711472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:49.731139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:49.732262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:56:49.733799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:49.736558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 241/488] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:56:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:49.744456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:49.752018 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 248/489] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:56:49.776058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:49.798058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:49.864290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:49.916570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dc3bd-1883-48a0-9328-23d645a2a71e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.923099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84cbb97-1879-4eaf-b559-1e65c2818ebe None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:49.957725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.958973 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.959427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.960038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.961057 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:49.961638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:49.963500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654a80c-9d7a-44da-b422-a2cdd5f4e888 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:49.972863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:49.972863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:49.972863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:49.972863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:49.985391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-998b17d0-c13c-4d0e-9c2d-d9c1be765b28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:49.995788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:49.997141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:49.998264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:50.001579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 249/490] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:50.004571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9042b-7020-421d-8a30-a322276be418 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.005171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9042b-7020-421d-8a30-a322276be418 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.005755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9042b-7020-421d-8a30-a322276be418 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.012519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 250/491] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:50.014646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9042b-7020-421d-8a30-a322276be418 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.031084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 242/492] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:56:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:50.033032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.033561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.034021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.055911 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:50.057578 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:50.116845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.142422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 243/493] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:50.145239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.190479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9042b-7020-421d-8a30-a322276be418 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.217401 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.230475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.231154 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.231445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.231727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.232518 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:50.232842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.238874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.249841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 251/494] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:50.253123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.278920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e10a2f-e7aa-4244-b231-97c3d1d56736 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.318637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.319943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.320237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.320548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.386436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f892-6b17-48eb-8914-86d6afb55f79 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.393466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.395018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.432157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.432989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.433274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.433554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.435634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.447343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:50.452637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-632703dc-c2bc-4736-9c14-4a1c2d5498c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:50.455508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:50.456915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:50.458619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:50.461726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 252/495] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:50.463109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc851-c9a7-4ba3-8dac-851710cfc51d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.463406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc851-c9a7-4ba3-8dac-851710cfc51d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.463689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcc851-c9a7-4ba3-8dac-851710cfc51d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.465031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 253/496] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:50.465717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcc851-c9a7-4ba3-8dac-851710cfc51d None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.523427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 244/497] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:50.523427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.603602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcc851-c9a7-4ba3-8dac-851710cfc51d None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.642496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59c192-86b2-4450-88b1-774620be8465 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.648219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.649060 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.649353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.649637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.663017 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.664550 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8d2511ea-247f-4dcb-8128-01527ce3db48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:50.682757 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:50.683821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:50.684827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:50.684827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:50.684827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:50.684827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:50.684827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 254/498] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:50.685435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.685618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.685904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.689545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.690556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.691126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.691595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.692787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:50.693280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.696779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.705976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:50.707149 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:50.712946 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 245/499] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:50.714698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.729975 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.754507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.781755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 255/500] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:50.782609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.838075 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d21b3-0171-41c1-99bc-ee259fcfe6f8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.874256 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.875059 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.875344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.875623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.886620 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.888003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44df061-a45f-4cc5-b7a1-4269f1635e49 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:50.888155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:50.905846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:50.906874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:50.907894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:50.907894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:50.907894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:50.907894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:50.907894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 246/501] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:50.909483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.929725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:50.930407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.930698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.930982 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.931717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:50.932033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:50.933760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2600a6-2866-4791-a49c-6d3d4c831b07 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:50.946010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 256/502] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:50.947091 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b850-f2a8-4fa6-aad0-8ac547d37eee None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:50.947383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b850-f2a8-4fa6-aad0-8ac547d37eee None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:50.947668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4b850-f2a8-4fa6-aad0-8ac547d37eee None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:50.949133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 257/503] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:50.949669 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b850-f2a8-4fa6-aad0-8ac547d37eee None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:50.976676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:50.987937 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.019738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fff4b-a9de-4800-9feb-9a805f0027e9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.059288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4b850-f2a8-4fa6-aad0-8ac547d37eee None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.059514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.060124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.060441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.060722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.080473 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.081961 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.096516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.097325 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.097611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.097887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:51.101598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.102532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.103575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:51.103575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:51.103575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:51.103575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:51.103575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 247/504] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.104157 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.114100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:51.122439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.123631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:51.124592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 258/505] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.125353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.215423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff4012-bd34-4557-8c15-52bfdc9574f7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.230197 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.242579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.248717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf0dd0-17e3-4cd2-9c76-0aa76b029110 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.252082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.252779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.253067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.253346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.254069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:51.254377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.255766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b81d8-f5db-4223-89a1-98ac5d486630 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.268737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 248/506] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:51.269824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.270115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.270411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.288847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.289658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.289944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.290224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.296512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:51.298003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:51.298565 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.300037 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:51.318415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.320225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 259/507] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.322053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.376873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 249/508] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:51.377812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.445441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2478-727a-4239-8966-cfc49de4c1a6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.482416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757330a-a4e4-46e0-ad67-87e7dd4ddff1 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.485245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.485571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.485948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.486241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.486522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.487286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:51.487747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.489590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.502776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 260/509] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:51.503610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.523356 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.524040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.524605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.524703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.525396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:51.525712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.525846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.527194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.539926 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 250/510] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:51.540948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.620522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a538ed-dc40-4b28-82c9-45c0e6a29290 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.656897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026257c-264a-442e-8078-70799cd8a00f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.657441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.658245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.658529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.658799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.665985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.667455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:51.682672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.684637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 261/511] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.686805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.699179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.699873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.700165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.700541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.701303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:51.701618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.703075 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.721301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 251/512] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:51.726758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.754237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.781143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:51.807329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472e7ba-d0e9-41ec-ab33-f2393c8bc648 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.835821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79bb3e9-17e8-4429-b994-c0790cc63520 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.841615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.842412 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.842697 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.842974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.854563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.856047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.871214 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:51.872002 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:51.872278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:51.872603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:51.874152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.875281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 262/513] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.876432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.886047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:51.887502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:51.902290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:51.903603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 252/514] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:51.904933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:51.990607 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f665b999-ed7d-467b-9404-7467cd092f44 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:51.997942 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.021137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.026001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863a40-dbd9-46ed-9d00-c3d1b54341d8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.031047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.031861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.032150 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.032457 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.045135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.046595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.062994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.063665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.063952 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.064228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.065058 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:52.065381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:52.065570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:52.066510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:52.067617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:52.067617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:52.067617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:52.067617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:52.067884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.067968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 263/515] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:52.067968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.086105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 253/516] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:52.086956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.184468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aa4f6-ed86-49bb-a8f7-4934052b4c90 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.198301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23269b68-9cd8-4877-bd6e-10ec74e94f46 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.218868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.219534 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.219916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.220091 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.234212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.234879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.235168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.235448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.236168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:52.236503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.237900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.252080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 254/517] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:52.252950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.257777 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.279201 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.301055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.302559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.369183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039227dd-68a4-4b8f-b399-4a22de2ff666 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.407571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.408241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.408562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.408839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.409670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:52.409978 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.411422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.424683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 264/518] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:52.424683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.441090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 255/519] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:52.441951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.499914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.510128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.521613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b9753-0966-4816-a5ab-9f4a6e17522b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.557757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5c7ba-33a8-4a37-a5ec-9cb4845f03c0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.559827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.560537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.560827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.561108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.561840 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:52.562148 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.563532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.575525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 265/520] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:52.576368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.599453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.601439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.601948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.602458 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.603551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:52.603870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.605327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.610876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 256/521] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:52.611662 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.722301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340574b3-300f-4d13-964e-cbea995916fc None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.744539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1c0ec-a2d3-41ac-8724-ca1ceba5ea5f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.766160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.766850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.767135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.767409 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.768135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.769671 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.775151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.781600 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:52.792171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:52.793738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:52.794029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:52.794305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:52.809268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 266/522] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:56:52 2022] POST /identity/v3/users => generated 247 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:56:52.810736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.820652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:52.822193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:52.854957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:52.856111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:52.857130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:52.857130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:52.857130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:52.857130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:52.860999 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 257/523] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:52.863198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:52.963664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c57247-9884-4352-9a6f-1bd59c77fb20 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:52.999876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391e8b-56d6-474d-96dc-158d0b749f22 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.008699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.009430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.009721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.010051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.039583 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.040389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.040706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.040986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.051465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.052986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:53.074780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:53.076728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:53.077917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 258/524] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:53.084017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.099068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.108262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.110410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.143324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.213451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f670a6-ab1a-4b8f-a2b2-9455608a3a05 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.248861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.249552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.249845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.250184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.250871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:53.251178 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.252618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.258971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 267/525] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:53.265221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.270460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 259/526] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:53.270756 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.380322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.382805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.388143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9db3f-cb53-40d2-9644-1a7459d71a91 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.389301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69187d-fa04-4cf3-869c-649ed627914d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.425883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.426684 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.426969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.427243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.429096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.429962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.430399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.430825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.431835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:53.432300 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.433956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.438835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.440273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:53.459453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:53.461406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:53.462592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:53.462592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:53.462592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:53.462592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:53.466325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 260/527] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:53.467307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.479423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 268/528] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:53.480684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.580885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837b477-e17a-47f9-b4ca-6b18d5058f51 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.594624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d69b4-0ed5-41a6-9421-540010d07ee2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.618892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.619714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.619996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.620269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.628490 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.629956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.639389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.640590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.641224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.641885 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.643227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:53.644021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:53.645190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:53.647033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:53.649562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:53.651674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 261/529] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:53.652343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.652618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.666877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.672572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 269/530] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:53.674254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.692950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.771167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f96616-ae98-4660-8a12-782d1e2f684a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.797548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98163db0-4696-4cf9-b5b9-6299f78f3ec2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:53.809337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.810147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.810302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.810540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.839913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:53.841087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:53.841887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:53.842331 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:53.843388 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:53.843962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.845619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.866749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 270/531] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:53.867877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:53.892630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:53.894021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:53.906014 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.959046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:53.992266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bac93d-5655-4c11-abc9-b0145da875f4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.003321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 262/532] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:54.004245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.032445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.033851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.034353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.034802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.044061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.045884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:54.064257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:54.065598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:54.066826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:54.073567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 271/533] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:54.074811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.111724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ff28f-ed5a-4cde-b582-e2f3bd84d08c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.123413 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:54.150431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.151020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.151312 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.151582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.152460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=1db16fe694454ce3b699c04679074feb) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:54.152781 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.154282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.168533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 263/534] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/role_assignments?user.id=1db16fe694454ce3b699c04679074feb&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:54.169557 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.201315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d3ec5-4147-41af-92ae-6339a5b10c96 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.243597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.244800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.245293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.245743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.248347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:54.260011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.261954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:54.282732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:54.284026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:54.285276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:54.285276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:54.285276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:54.285276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:54.288445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 272/535] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:54.298402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.392329 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc152d0b-98bb-4662-b31d-504cc6fc2ceb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.451889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.452533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.452820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.453093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.453904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:54.454212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.455664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.471785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 264/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:54.478871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.519584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5575aaa-b171-4d85-890b-fc52611fe952 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.524012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:54.575693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.576376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.576684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.576960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.577699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:54.578010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.579402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.603859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 273/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:54.609665 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:54.687910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2f791-d108-447b-88d8-5f35c6b93481 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.725210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:54.786192 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.787167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.787902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.788346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.790928 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=67ac4e9b064e44c3aba497e5e7ecc838) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:54.791253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.792746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.809000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 265/538] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/role_assignments?user.id=67ac4e9b064e44c3aba497e5e7ecc838&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:54.847871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab91-d036-4ce7-b115-535c3112fb70 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:54.908873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:54.910224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:54.910775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:54.911323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:54.923402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:54.925554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-39a0a841-2d36-407c-b366-a7be5f61f4ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:54.947632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:54.948779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:54.949774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:54.953616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 274/539] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:54.960021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:55.105332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1513b-9b4a-4215-8da0-a66f3eef4e79 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:55.108718 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:55.129140 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526c38-7a66-4996-92f2-133c1582cd9e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.129831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526c38-7a66-4996-92f2-133c1582cd9e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.130377 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86526c38-7a66-4996-92f2-133c1582cd9e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.138086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 275/540] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:56:55.143557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.144147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.144696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.156296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:55.157326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.160469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.160469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.160469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:55.160469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:55.161770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:55.182810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 266/541] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:55.186881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:55.191878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:55.192698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:55.353002 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 276/542] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:55.364427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0de2d7-75b3-40ad-b8e5-e132b55711e4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:55.379679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:55.435343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:55.436011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.436301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.436612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.437479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=82dd081413a44b77b8d1342f2755ce51) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:55.437810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:55.439330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2490b-d29b-4e08-a5ce-c3142551f14f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:55.447577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 267/543] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:55 2022] GET /identity/v3/role_assignments?user.id=82dd081413a44b77b8d1342f2755ce51&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:55.492798 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:55.512090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671d580-f07b-45f6-91a0-8698051b7645 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.512489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671d580-f07b-45f6-91a0-8698051b7645 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.512688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671d580-f07b-45f6-91a0-8698051b7645 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.514121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 268/544] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:55.523393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.524007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.524560 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.549525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:55.551589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:55.568783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3fa50-d1a3-4bd0-bdee-229cdaf6aee7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:55.596974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:55.659840 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 269/545] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:55.660079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:55.660761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.661046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.661321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.678847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:55.686585 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:55.691162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:55.692298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:55.693311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:55.698746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 277/546] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:55.705448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:55.895646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e84dfd-158c-4fbc-b65c-5a4014347ff2 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:55.899920 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:55.913965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2a1ab-3a86-44d2-b473-5ff5d9f0d0fb None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:55.968430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:55.970822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.971523 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.972831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.983127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:55.984126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:55.984683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:55.985213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:55.986184 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:55.987449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:55.989302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:55.996924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:56.010777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 278/547] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:56.012354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:56.013395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:56.014349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:56.017851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.027096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 270/548] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:56.029600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.066400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:56.185355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5f82a-3adc-42f9-94e7-17cf7b3bce10 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.251293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766947-737b-4eb4-9134-05fe185e502b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.257972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.258957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.259700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.260218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.265392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:56.265987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.267623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.290621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 271/549] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:56.293208 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.321314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.322865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.323410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.323930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.343501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.345498 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:56.371844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:56.372931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:56.375167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:56.375167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:56.375167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:56.375167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:56.375167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 279/550] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:56.382852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.407820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:56.464861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:56.510487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2a9fc-0c11-44e4-b2e1-532358c346a6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.577636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.578444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.578725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.579001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.585506 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f4a87-cb86-426a-9f9c-5ca71ac165fc None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.603596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.605105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:56.643552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:56.644647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:56.645806 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:56.645806 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:56.645806 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:56.645806 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:56.648492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 272/551] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:56.655089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.679564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.680778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.681298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.681804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.683206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:56.683778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.686456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.728461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 280/552] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:56:56.735763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.851794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:56.887256 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d84976-24f8-4fb4-8be1-bf9318571271 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.913276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b08a7c-2d6f-4b61-87da-53374bcf8478 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:56.937551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.938354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.938639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.938918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.945078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:56.949618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.951000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.955351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:56.956364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:56.957113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:56.957632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:56.958717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:56.959275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:56.961743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:56.980314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 273/553] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:56.981939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:56.984066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:56.985203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:56:56.986501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:56.992697 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 281/554] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:56.993712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:57.109188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a285a-8b90-47d8-8ec0-de37a79515ed None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:57.149089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:57.150448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.150969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.151468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.159969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:57.162132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:56:57.180345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:57.181489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:56:57.182640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:57.190947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 274/555] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:56:57.192549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:57.250439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:57.287044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b28e6-1563-4770-9a12-db2ac0d9f5d9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:57.315468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e924e-9c43-478c-a13c-31028a206694 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:57.359682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:57.362012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.362550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.363065 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.364365 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:57.366204 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:57.368690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:57.400239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:57.407477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 275/556] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:56:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:57.411080 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:57.412130 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.413231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.413736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.414966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:57.416527 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:57.417629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:57.421251 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d18f3-0450-4883-840d-b850e5445bde None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:57.450657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 282/557] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:56:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:57.451898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda507-3406-44a1-8e66-96ece4ef9389 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.452200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda507-3406-44a1-8e66-96ece4ef9389 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.452541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eda507-3406-44a1-8e66-96ece4ef9389 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.458258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 283/558] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:57.459339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eda507-3406-44a1-8e66-96ece4ef9389 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:57.554215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92feed7-4b4c-4bf2-ad47-602d271cc74d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:57.557707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:57.596618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:57.597306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.597600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.597882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.603221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eda507-3406-44a1-8e66-96ece4ef9389 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:57.664051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:57.665365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.665983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.666328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.682189 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:57.683694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:57.684672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:57.686137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da994dc-39d9-407c-9491-58b1465761a6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-d2c30431-858b-4f40-bf7f-01a757f4dbed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:57.713070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:57.714207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:57.715298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:57.715298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:57.715298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:57.715298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:57.715298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 284/559] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:57.715705 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744311a-b4dd-4825-9030-e0059991d2f9 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.717633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744311a-b4dd-4825-9030-e0059991d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.717633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744311a-b4dd-4825-9030-e0059991d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.735174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b744311a-b4dd-4825-9030-e0059991d2f9 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:57.736504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b744311a-b4dd-4825-9030-e0059991d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:57.778717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:57.831598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 285/560] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:57.831598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c6ac-68c5-4b3a-8be8-304212de9aa8 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.831598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c6ac-68c5-4b3a-8be8-304212de9aa8 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.831598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4c6ac-68c5-4b3a-8be8-304212de9aa8 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.833744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 286/561] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:57.834767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c6ac-68c5-4b3a-8be8-304212de9aa8 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:57.837790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 276/562] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:56:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:56:57.839604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:57.839996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:57.840529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:57.868393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:57.888213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:57.890326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:57.991801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 277/563] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:57.992807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.023988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4c6ac-68c5-4b3a-8be8-304212de9aa8 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.080127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.080744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.081034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.081315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.082066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:58.082379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.084090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.102057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 287/564] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:58.102057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.135356 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f15a45-6558-494f-89bd-436e698f8d18 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.144691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.181008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.182210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.182605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.182802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.185833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.197753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.199743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:58.228081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:58.229302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:58.230349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:58.230349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:58.230349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:58.230349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:58.230349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 278/565] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:58.230651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.248720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b88b7c-34eb-4954-bdc5-fdf9979ee802 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.293846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.294658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.294950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.295229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.306651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-cc77a681-339d-4c74-9277-1944ed40c5a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:58.315767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:58.317061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:58.318072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 288/566] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:58.319421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2d5d-a490-43eb-a82b-4dbce2012563 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.319421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2d5d-a490-43eb-a82b-4dbce2012563 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.319665 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd2d5d-a490-43eb-a82b-4dbce2012563 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.320897 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 289/567] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:56:58.321630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd2d5d-a490-43eb-a82b-4dbce2012563 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.360515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7126ac-748c-4613-b811-73c84eea5e64 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.412424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.413253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.413538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.413820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.424844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:58.432458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:58.438316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:58.439495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 279/568] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:58.440616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.450479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd2d5d-a490-43eb-a82b-4dbce2012563 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.495334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.496822 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.497741 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.498105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.498495 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.507391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.509493 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:58.530270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:58.531418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:58.532729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 290/569] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:58.533731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.563700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c0d8-96a3-4bd5-8501-502592ff9176 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.603072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.603737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.604037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.604462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.605766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:58.606311 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.608728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211f988-3902-4ca3-a8a3-0f4f7fae16e3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.627117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 280/570] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:58.628439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.628831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.629341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.655976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:56:58.656370 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85d7fe-2a17-4c8b-bb22-8ca2bef4a803 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.657243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:56:58.692380 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.693106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.693413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.693666 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.695396 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:58.695818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.697438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.706548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 291/571] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:58.707409 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.724915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.730860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 281/572] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:56:58.731714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.768589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.830678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3a23a-71e4-4016-aff7-b5177c875416 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.838433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b4ccd-0c00-4f9d-a0f8-3684aa9d7a0e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:58.866438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.867112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.867403 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.867676 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.868450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:58.868762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.870202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.878977 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 292/573] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:58.879732 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.897792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:58.898779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:58.899138 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:58.899482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:58.900366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:58.900709 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:58.902237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:58.914989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 282/574] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:58.915812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:58.962880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:58.989548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87ce9b-3057-493e-8d0a-66cb323e7c8e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.016350 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.023476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55804008-cb0d-4492-a8f6-39ace290f7bc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.028701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.029511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.029797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.030076 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.039084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.040596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.058369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.059172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.059485 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.059764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:56:59.064296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:59.065553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 293/575] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:59.066546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.072000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.073515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:59.092164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:59.093436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 283/576] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:59.094586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.183214 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8131708-bf46-4e14-90b0-8079b14ec0a2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.202843 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309859-2373-4bc6-966d-e92b856e426e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.219733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.220579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.220869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.221150 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.223100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.233950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:56:59.240047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.240908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.241198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.241478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:56:59.241902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:59.243069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 294/577] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:59.244455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.254474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.255928 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.264912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:56:59.275935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:56:59.277432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:56:59.278632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:56:59.278632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:56:59.278632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:56:59.278632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:56:59.281953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 284/578] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:56:59.282901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.358843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb21cb5-2f2e-4321-868f-1f8c048c69d6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.392208 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27a6-71f0-4ba9-80b3-617e8d94291b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.398915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.399804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.400088 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.400357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.416127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=482a2286f928483b8bce71f07fb4809f, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.417961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.432187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 295/579] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:56:59 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:56:59.433078 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.435330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.435958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.436242 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.436559 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.437361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:59.437674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.439163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.454793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 285/580] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:59.455567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.500583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.543564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.582715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ee54-d440-4aa8-b7ee-11849af8579e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.597444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1b291-a0d5-471d-b946-33b2c51cfb87 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.629638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.630479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.630786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.631132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.632500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:59.632807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.634522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.646820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.649502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.650019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.650496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.650621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 296/581] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:59.651519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.651859 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:59.652347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.653944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.664232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 286/582] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:59.665383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.770269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef78846-59d8-4013-9f4f-83f0d6b6c8c5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.773488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.799395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc930b4-72e3-4d6c-9e6f-02654b119a10 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.816043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.816759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.817038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.817317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.818049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:59.818363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.820426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.827688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:56:59.837231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 297/583] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:59.838660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.847280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:56:59.847998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:56:59.848285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:56:59.849251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:56:59.850260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=af1926a9c0e146219e85e0021a489b59) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:56:59.850783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:56:59.853286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:56:59.865948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 287/584] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/role_assignments?user.id=af1926a9c0e146219e85e0021a489b59&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:56:59.865948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:56:59.978106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031637c-0061-40c9-b8f3-2b66cf2db85e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:56:59.999540 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25d72-adbd-4de5-b714-338b82f1de2a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.020152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.021327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.021662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.022623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.030962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.032603 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.043485 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.044438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.044805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.045158 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:00.050524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.051829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:57:00.053231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:00.058861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 298/585] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.060044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.060605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.062170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:00.082121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.083374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.085687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:00.085687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:00.085687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.085687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:00.085687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 288/586] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.089273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.133999 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.144750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.181876 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf317-5885-421b-8ded-6b78fa863ee5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.201870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d39cd-199f-4077-8bc8-5f3a8348124a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.222735 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.223938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.224461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.225105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.233902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.235848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.240697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.241386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.241677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.241959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.242807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:00.243124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.244666 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:00.253712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.254831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:00.255864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:00.259750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 299/587] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.261131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.270300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 289/588] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:00.270300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.382834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.385621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.394412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aba12e-921e-467e-b5db-bcfe134bbe27 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.405433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410ed3-8284-4d06-8428-511ef558260f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.432878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.433716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.434098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.434472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.435413 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:00.435846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.437418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.458308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.458766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 290/589] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:00.459960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.460242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.460910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.461368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.462336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:00.463015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.465129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.487473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 300/590] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:00.487473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.587046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db385cb3-88a2-4b33-a096-4c355c6e3e50 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.605841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42db9f7-be0c-4aab-be01-474da4db5c03 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.627488 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.628593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.629061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.629570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.643374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.645592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.645832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.646770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.647782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.648128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.663206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.664818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.667748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:00.671377 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.673098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.674189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:00.674189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:00.674189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.674189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:00.677462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 291/591] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.679664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.692361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:00.692583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.693557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 301/592] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.695908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.807258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da77274-d983-425d-b3ae-428ac331b3f8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.813190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8b1b6-5d99-45be-aaaa-ac3c5eea9fec None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.846497 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.847403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.847689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.847965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.852571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:00.853239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:00.853525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:00.853803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:00.854531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:00.854836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.856226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.856824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:00.858362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:00.873842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:00.875636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:57:00.876754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:00.877517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 302/593] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:00.877517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.877672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 292/594] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:00.877672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:00.939768 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.952130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:00.990119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e3b5-341f-4590-92fa-c6474c49f3f5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:00.992719 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b65a5-c47e-4790-b3b1-7be07e5ff77a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.029324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.030021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.030274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.030551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.031273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:01.031832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.032761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.032968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.034455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.034954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.035421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.045525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.047406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 303/595] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:01.048132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.048576 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:01.065043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:01.066276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:01.067416 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:01.074727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 293/596] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:01.075882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.168490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029b6fa-e947-4b98-84a8-2550217ceae0 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.187353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.194548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5bd73-86a0-4be5-ac44-559efaa39987 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.214172 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.214860 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.215151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.215565 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.216385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:01.216922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.218658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.226385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.237896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.238977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.239428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.239869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.240447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 304/597] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:01.241833 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.249186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.251227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:01.267547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:01.268833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:01.269974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:01.270449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 294/598] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:01.270960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.363104 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379720a8-d704-4854-ac4e-f81df81f081c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.388490 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815042ac-351d-481e-9834-933961ad4f22 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.402414 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.403079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.403362 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.403641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.404474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:01.404791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.406602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.425980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 305/599] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:01.425980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.434893 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.436058 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.436577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.437002 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.451189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.453176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.461111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:01.477409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:01.478750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:01.479820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:01.479820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:01.479820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:01.479820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:01.481849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 295/600] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:01.482847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.538083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.539229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57822d68-d66d-4bda-a9fb-af2b5c0f1efe None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.580226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.580953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.581242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.581517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.582316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:01.582627 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.584217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.603995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 306/601] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:01.605084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.611782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9714d450-491f-477e-a655-13508e07dc56 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.652590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.653877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.654325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.654753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/1db16fe694454ce3b699c04679074feb/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.674482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=1db16fe694454ce3b699c04679074feb, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.676574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.700703 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 296/602] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:01 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/1db16fe694454ce3b699c04679074feb/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:01.702070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.719584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.729330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db527bd-c1b3-436e-ac9c-a853862fea09 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.776607 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.777373 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.778787 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.779114 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.779896 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:01.782015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:01.783575 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:01.814351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 307/603] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:01.821652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:01.903938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:01.922754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5cf10-9aed-4c1d-9edb-345319d45d8b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:01.976676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:01.977831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:01.978137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:01.978427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/67ac4e9b064e44c3aba497e5e7ecc838/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:01.999367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=67ac4e9b064e44c3aba497e5e7ecc838, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:02.001454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615d930-61fa-4fdc-949c-5b56599ebd0b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:02.002332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bfadd2-ba0e-41bc-bc78-110dfa58c577 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.020522 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 297/604] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:01 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/67ac4e9b064e44c3aba497e5e7ecc838/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:02.022343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55485-4157-4805-9634-cbc9a3106d49 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.022689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55485-4157-4805-9634-cbc9a3106d49 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.022978 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55485-4157-4805-9634-cbc9a3106d49 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.029955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 298/605] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:02.032049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.032617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.032915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.055699 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:02.057066 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:02.083226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.084067 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.084381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.084749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.109955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:02.111488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:02.145477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:02.148682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:02.151212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:02.160436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 308/606] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:02.160436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.160993 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 299/607] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:57:02.196803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:02.224118 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.333493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:02.355668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1710b0-4642-46f9-9e7c-02db61c77e7e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.382219 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99826a67-b26d-4a1e-8447-cc64c54b7f1a None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.398204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.398961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.399251 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.399573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.400485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:02.400805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:02.402303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b9a26-9130-490b-b0bb-8a1e571524ec None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:02.416419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 309/608] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:02.417781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae337d-ced8-49a9-80ae-cf5242842fd7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.418076 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae337d-ced8-49a9-80ae-cf5242842fd7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.418361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae337d-ced8-49a9-80ae-cf5242842fd7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.419888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 310/609] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:02.425000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae337d-ced8-49a9-80ae-cf5242842fd7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.430502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.431321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.431608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.431887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.446927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e6d70f72-2494-4adf-a425-29a7fdaca0bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:02.457275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:02.460318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:02.461432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 300/610] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:02.462364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.462502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.463040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.490590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:02.492788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:02.523504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:02.555856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae337d-ced8-49a9-80ae-cf5242842fd7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.585137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 301/611] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:02.586171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.608559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.609535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.609828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.610108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/82dd081413a44b77b8d1342f2755ce51/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.632021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=82dd081413a44b77b8d1342f2755ce51, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:02.634394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:02.655350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 311/612] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:02 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/82dd081413a44b77b8d1342f2755ce51/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:02.656662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.675042 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:02.743002 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40532529-6035-4ac6-8250-c5534c1d22d0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.795254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.795775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e058a3-e2b2-42a3-816c-4b46baae79c7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:02.795992 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.796229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.796539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.797349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:02.798238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:02.800886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:02.814138 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 302/613] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:02.823809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:02.845042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:02.862566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:02.863846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:02.863846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:02.863846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:02.878919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:02.888563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:02.889663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:02.890597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 312/614] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:02.895769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.012940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f729e-c1c1-47e8-958f-7d8b0e9e1b69 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.031593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997bfd-1bab-4735-9e1b-0a25d4d84569 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.042689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:03.066996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.070543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.071547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.072073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.076229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.077253 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.077589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.077869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.078638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:03.078945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.080352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.087901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.090017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.090832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 313/615] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:03.099137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:03.111546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:03.112639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:03.113629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:03.113629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:03.113629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:03.113629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:03.113885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 303/616] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:03.122921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.217125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:03.265381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10ff14-17f6-4a65-9224-8450a37e9383 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.305317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.306273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.306577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.306860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.307603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:03.307918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.309945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.324647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 304/617] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:03.333753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.340139 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316a1cc-0be2-41e0-9d3c-1c7929be6345 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.378500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.379235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.379520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.379799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.400776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:03.405938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.407431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:03.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:03.449608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:03.450623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:03.450623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:03.450623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:03.450623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:03.453867 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 314/618] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:03.458027 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.556819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ad759-903f-470b-8450-57d00d085dc5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.635333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.636052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.636337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.636638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.656884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.658386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.678725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:03.682901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8dac4-d0ab-48a9-ae56-615d3c1fc25b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:03.703096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:03.705641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:03.708173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:03.711639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 305/619] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:03.721431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.735228 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:03.736020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:03.736375 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:03.736759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:03.737574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:03.737956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:03.739906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:03.755226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 315/620] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:03.760984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:03.903401 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:03.927596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab2324-ac2b-4cd7-856b-298a4c94dadd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:03.930520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d5a0b-fb45-4e6a-b74a-94545c640012 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:04.020889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.020889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.020889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.020889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.021359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:04.021359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:04.021359 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:04.042319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.048422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.048422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.048422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.058510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 306/621] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:04.070701 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:04.081001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:04.082566 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:04.125246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:04.126786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:04.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:04.134126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 316/622] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:04.139911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:04.199783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:04.444784 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af219482-0e1e-41c2-a22b-fc87b0b5b923 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:04.510901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb12b42-445a-471c-9329-b88233decac1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:04.529855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.530584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.530879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.560343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.561980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.562618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.563131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.564540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:04.565502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:04.567616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e41cf-6b85-43ea-8b67-98bd5b7361f9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:04.568607 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:04.593509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 317/623] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:57:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:04.595223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c2ac-89b6-44a0-8cbc-1e62177a95f7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.595815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c2ac-89b6-44a0-8cbc-1e62177a95f7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.596317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3c2ac-89b6-44a0-8cbc-1e62177a95f7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.598271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 318/624] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:04.600851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3c2ac-89b6-44a0-8cbc-1e62177a95f7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:04.642664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:04.644606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25087f0-51be-4db0-bfd9-3e210b69facf None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:04.711835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:04.737955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3c2ac-89b6-44a0-8cbc-1e62177a95f7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:04.764171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 307/625] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:04.765771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adbf15-c1cc-400f-a124-e56e46e8099c None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.766227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adbf15-c1cc-400f-a124-e56e46e8099c None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.766657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7adbf15-c1cc-400f-a124-e56e46e8099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.781419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.782457 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.782858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.783268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.785338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-f7adbf15-c1cc-400f-a124-e56e46e8099c None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:04.786917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-f7adbf15-c1cc-400f-a124-e56e46e8099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:04.796204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:04.798157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:04.819357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:04.820458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 319/626] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:04.821459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:04.846394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:04.871354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 308/627] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:04.873378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82dd5-135f-40bd-a557-6c408feb739a None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.873910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82dd5-135f-40bd-a557-6c408feb739a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.874406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb82dd5-135f-40bd-a557-6c408feb739a None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:04.880497 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 309/628] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:04.881996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82dd5-135f-40bd-a557-6c408feb739a None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:04.948924 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690cbed-efb1-47af-b4f9-b48df75250ac None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:04.978735 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:04.987235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:04.988042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:04.988328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:04.988636 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.001690 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.003185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.008764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb82dd5-135f-40bd-a557-6c408feb739a None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-29d80910-123b-4b9b-a9ff-eb607ab235a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:05.025850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:05.026878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:05.027987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:05.027987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:05.027987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:05.027987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:05.027987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 320/629] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:05.028307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.030273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.030273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.049835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:05.051258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:05.055293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.055977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.056259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.056562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.057293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:05.057603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.058989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.076774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 310/630] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:05.076774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.129670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.144912 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 321/631] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:05.144912 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.212018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f210be9-c36a-45cd-b389-174f7e3aa8b3 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.260295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.260987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.261297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.261538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.271285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d9e69-6104-4276-ac0a-f51f12060c42 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.273188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:05.281238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:05.282338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 311/632] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:05.283395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.290418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.325112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.325790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.326081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.326360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.327097 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:05.327409 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.328824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.340784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 322/633] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:05.341620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.395678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.399061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbfdf4-5be1-41d8-a60a-eef6cb3bd66b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.435182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.435994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.436285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.436596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.455635 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ec083-91f6-45af-a1c8-756c465a610f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:05.460810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:05.462018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 312/634] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:05.463254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.496677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.497478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.497765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.498047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.509143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c5cd5148-d6d7-4f67-83da-668d84dca8c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:05.517132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:05.518304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 323/635] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:05.519765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c3d21-cc3b-4e72-a9d3-78ca038063a5 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.520639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c3d21-cc3b-4e72-a9d3-78ca038063a5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.520639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c3d21-cc3b-4e72-a9d3-78ca038063a5 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.521610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 324/636] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:05.522661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c3d21-cc3b-4e72-a9d3-78ca038063a5 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.556229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.603266 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae985-4dee-4bf6-9389-af621af0e5f9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.633924 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c3d21-cc3b-4e72-a9d3-78ca038063a5 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.640077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.640786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.641071 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.641343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.642091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:05.642400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.643765 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.652846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 313/637] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:05.653663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.682951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.683792 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.684039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.684312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.685385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.691449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.692962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-3e223cea-80f4-4050-94e7-88b3dab9838b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:05.708751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:05.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 325/638] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:05.711966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.712765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.712765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.729409 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:05.730631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:05.771408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2bea6-0184-4246-9aaf-ca2cfb15ce08 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.799865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 326/639] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:05.800770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.809065 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.809653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.809938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.810212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.810997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:05.811304 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.812757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.823790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.827207 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 314/640] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:05.828093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.906721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e70e-545c-435c-b62f-5bceb63c2a23 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.920982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:05.940855 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496add5-b286-46cc-8489-dc908a558aef None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:05.946586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.947262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.947549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.947826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.948681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:05.948992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.950386 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:05.958564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 327/641] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:05.959958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:05.979026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:05.979816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:05.980095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:05.980372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:05.991888 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:05.993417 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:06.011596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:06.012825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:06.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:06.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:06.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:06.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:06.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 315/642] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:06.014206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.058680 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.078506 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb021-869e-4e6c-b30c-d4040814b499 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.115521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.116218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.116535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.116812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.117638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:06.117945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.119392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.130172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb7383-fd5e-4922-91f1-6d06313a3877 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.131512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 328/643] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:06.133394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.175528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.176327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.176639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.176912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.179510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.189857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:06.199046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:06.200726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 316/644] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:06.201668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.246868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4ca1f-7de0-4e78-b973-d360ba4eed80 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.282152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.283049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.283333 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.283611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/af1926a9c0e146219e85e0021a489b59/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.297243 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.301972 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=af1926a9c0e146219e85e0021a489b59, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.305168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.312904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3a1c5-dae5-4e8c-b333-f6c897e675f7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.320470 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 329/645] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:06 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/af1926a9c0e146219e85e0021a489b59/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:06.321315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.371507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.372233 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.372578 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.372860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.386316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.387817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:06.409507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:06.410632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:06.411625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:06.411625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:06.411625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:06.411625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:06.411625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 317/646] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:06.412193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.463050 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24dbf6-7b4b-4045-b873-131246182d16 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.478119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.529022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.529824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.530115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.530385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.531344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:06.531654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.533374 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.547496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436df0-743a-4d99-9ab0-b03ef7d2c5f5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.551890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 330/647] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:06.551890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.593946 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.594564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.595119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.595990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:06.596262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.597727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.612464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 318/648] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:06.612464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.623227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.684968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864f7b1-03ad-485a-8568-1baa4d5fdc5b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.728620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.729305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.729588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.729865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.730625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:06.730931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.732570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.743272 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 331/649] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:06.744159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.777661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee93f2a-ac1b-464e-a071-252e233e494b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.793624 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.831383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.832977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.833263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.833541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.834873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:06.835200 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.836672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.852551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 319/650] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:06.853489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.886089 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd78fab-fad5-436b-883f-326031bd68fb None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:06.903739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:06.932095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:06.933105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:06.933495 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:06.933857 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:06.942254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:06.943905 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:06.962554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:06.963700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 332/651] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:06.964794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:06.993548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aafd59-0f9c-49d2-95ad-eb576338f6e1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.042697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.043515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.043802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.044112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.060423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.062812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:07.085241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:07.086334 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 320/652] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:07.087330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.114553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533e3eb-86d7-467c-9d26-a240fbc00cf9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.122171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.160436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.161303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.161600 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.161891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.173544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.175302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:07.196116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:07.197278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:07.198314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:07.205582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 333/653] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:07.206786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.225893 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305362d5-d438-4e08-ae92-eff4cc837257 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.268730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.275874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.277143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.278020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.278765 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.280062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:07.281014 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.283144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.317119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 321/654] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:07.319291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.331991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b78da-d5f1-4ae8-83ed-b3f8b0f5c109 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.378900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.379682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.380094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.380384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.381824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:07.382210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.384067 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.404266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 334/655] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:07.404266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.435851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.465340 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a093a0-8208-4318-8dfc-8755e3aef12f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.529681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.530613 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.531079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.532000 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.533340 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:07.533842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.535976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.538255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.553965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960cf0b-8a29-43a3-a907-764d7807a6d5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.562264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 322/656] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:07.563369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.618820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.619490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.619778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.620056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.712844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363a4da-ea8f-4002-933c-9460aee17cc9 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.718689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.720465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.760668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.761245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.761530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.761807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.770036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:07.771490 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:07.773889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:07.789383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:07.790543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:57:07.791593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:07.795358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 323/657] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:07.796128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.832513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:07.842337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 335/658] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:07.843492 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:07.923608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d97b65-e9f4-47e7-9972-0d166cb480f8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.956573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494e037-20a9-4513-ba11-64875e9a5d7d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:07.970958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.971165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:07.971442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:07.971716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:07.998514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:07.999582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.000085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.000633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.001696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:08.002223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.004049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.026766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 336/659] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:08.027996 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.050826 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.056011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.057678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.089870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.144383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddb047-134b-486c-88c5-72b1e52a2f8a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.173725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 324/660] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:08.173725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.190200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.191399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.191906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.192449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.204515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.206322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:08.227354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:08.228419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:08.229435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:08.229435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:08.229435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.229435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:08.231873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 337/661] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:08.233105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.286506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.292800 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a96bff-b2c0-4b20-8b3c-38dc01fd0fa1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.329515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.329994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.330278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.330553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.353124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2d481-dcf6-4b19-b679-7ac1af6cabe7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.356317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.398976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.400197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.400743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.401265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.407608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.409129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.413723 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.415459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:08.439777 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:08.441176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:08.442178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:08.442178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:08.442178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.442178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:08.449934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 338/662] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:08.451171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.522328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.528602 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 325/663] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:08.529455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.583498 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf185d1-e65d-47d3-9d1c-f63d97ac085e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.622920 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.623968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.624501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.625029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.625986 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:08.626513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.628126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.645162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 339/664] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:08.647577 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.653481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219adb-39da-4647-8e2e-2ef7a2abd378 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.669513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.691938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.692655 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.692940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.693211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.694538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:08.694849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.696287 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.709136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 326/665] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:08.710591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.773439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe72d1e-935b-4b07-bef7-3a116b2cc8f8 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.778817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.811739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.812576 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.812864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.813134 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.825040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.826501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.830923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81995b9-eefb-4392-b572-b52135ea47fe None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-cc0fad1a-9e67-4cec-927a-1fd36dc69ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:08.846625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:08.848238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 340/666] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1f653-251a-4df4-9347-5ebb09f7ab5d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.850061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1f653-251a-4df4-9347-5ebb09f7ab5d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.850769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1f653-251a-4df4-9347-5ebb09f7ab5d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.857188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 341/667] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:08.858156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1f653-251a-4df4-9347-5ebb09f7ab5d None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.876247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b6443-b867-474c-aede-17076f016af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:08.877127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6443-b867-474c-aede-17076f016af7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:08.877381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6443-b867-474c-aede-17076f016af7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:08.877655 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b6443-b867-474c-aede-17076f016af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:08.886223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b6443-b867-474c-aede-17076f016af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:08.887679 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b6443-b867-474c-aede-17076f016af7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-474b6443-b867-474c-aede-17076f016af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:08.902799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:08.904003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 327/668] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:08.905015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b6443-b867-474c-aede-17076f016af7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:08.937123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:08.979240 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1f653-251a-4df4-9347-5ebb09f7ab5d None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.017325 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.018124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.018410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.018689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.021118 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b6443-b867-474c-aede-17076f016af7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.026928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.028332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.040345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4ee860f2-aa8b-4611-9072-f361021bbbb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:09.043785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:09.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:09.045821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 342/669] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:09.046371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.046918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.046918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.065246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:09.066485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:09.066792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.067461 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.067747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.068018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.068875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.069211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.070547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.083069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 328/670] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.085802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.141519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 343/671] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:09.142779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.212759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82b7c8-ddfa-4e6e-abd5-1e5fc6c8f9aa None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.226938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.248906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.249668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.249850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.250123 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.250905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.251215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.252665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.265995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 329/672] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.266881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.274799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1e4b1-86c2-442e-960c-5ff9f33ed5ac None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.310273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.310860 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.311150 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.311432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.311708 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.312232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.312578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.314138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2c9e2-4455-4cdf-80c3-76acef8af3cc None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.328178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 344/673] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.329576 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296e584-4820-4812-bc3f-4db6c4f0c8f7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.329877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296e584-4820-4812-bc3f-4db6c4f0c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.330168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296e584-4820-4812-bc3f-4db6c4f0c8f7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.331504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 345/674] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:09.332113 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296e584-4820-4812-bc3f-4db6c4f0c8f7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.377750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaede2-5619-44d7-9880-0652be6ed7ad None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.413771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.414561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.414841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.415121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.422384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.424117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:09.438936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:09.440015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 330/675] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:09.441081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.447646 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296e584-4820-4812-bc3f-4db6c4f0c8f7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.465816 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.485034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.485845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.486127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.486393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.497236 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:09.505085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:09.506693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 346/676] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:09.507766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.551602 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.556337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dc843-19d8-4fea-aaf9-ea941aa7fefe None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.592772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.593346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.593629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.593907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.594746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=1db16fe694454ce3b699c04679074feb) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.595061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.596646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecda19-7fc6-4a6e-a1e1-4707b59842d0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.603546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 331/677] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/role_assignments?user.id=1db16fe694454ce3b699c04679074feb&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.604940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.605336 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.605512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.630236 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:09.632206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:09.648782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1976395-1ce2-4377-8138-219432cb0997 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.693285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.694693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.694978 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.695259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.696141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=67ac4e9b064e44c3aba497e5e7ecc838) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.696491 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.698315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.716889 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 347/678] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/role_assignments?user.id=67ac4e9b064e44c3aba497e5e7ecc838&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.716889 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.775523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:09.859500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534ccd-de3c-46d6-b4b2-5ebbf079ceae None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:09.870901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 332/679] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:09.871117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:09.926169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:09.926438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:09.927107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:09.927107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:09.928801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:09.928801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:09.930038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:09.947389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 348/680] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:09.948451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.035180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.048698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5efc370-7f22-4c60-ad67-87d34bdc453e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.089432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e0a6-fa45-4617-b944-f3a772d76060 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.091146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.097110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.098246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.098847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.099394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.100805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:10.101364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.102991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.113966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 333/681] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:10.115180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.127906 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.128871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.129079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.129315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.141031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:10.149981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:10.151138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:10.152591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 349/682] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:10.157290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.242046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31838196-9c55-46f0-9ecf-a9e5835051fd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.269044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb1cce-94b6-4f4f-b7c7-6486f26531cd None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.284034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.285171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.285687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.286187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.287468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=82dd081413a44b77b8d1342f2755ce51) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:10.288182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.289985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.297826 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 334/683] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/role_assignments?user.id=82dd081413a44b77b8d1342f2755ce51&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:10.299393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.308040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.309394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.309786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.310249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.324567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.326067 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.327273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:10.348341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:10.349449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:10.356873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:10.356873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:10.356873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:10.356873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:10.356873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 350/684] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:10.357922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.390686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.485656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270d119-c994-4cf1-8eb8-18dcbcbae0c7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.539128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeccc949-bb57-46af-891f-281b1105569c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.559619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.560331 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.560666 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.560947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.561813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:10.562128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.563623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.584033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 335/685] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:10.589462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.595792 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.596510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.596804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.597086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.597828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:10.598584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.601462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.628555 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 351/686] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:10.634850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.721480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.741189 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:10.747392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e9358-80e9-4aa0-8661-dcb467f865c7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.769035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fa853-e85d-4c63-aade-6b4c1a083446 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:10.789415 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.790647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.791141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.791717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.804628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.806841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.808008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:10.808927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:10.809282 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:10.809700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:10.817743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:10.819851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:10.829625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:10.830686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:10.831722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:10.831722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:10.831722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:10.831722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:10.834985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 336/687] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:10.840939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:10.842335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:10.843358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 352/688] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:10.845112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.848011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:10.961602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff473f4-19d0-48eb-b0ef-11481342a115 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.010281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.010892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.011181 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.011460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.012319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:11.012663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.014126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.016135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:11.029685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697e3b4-9f8f-4a0b-b1db-7d98069cc9a0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.056749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 353/689] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:11.060047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.065078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:11.072844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.073976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.074500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.074993 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.076039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:11.076607 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.078203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.092960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 337/690] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:11.100567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.206634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0db32c-0000-49c5-8a53-378796e1642d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.255657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.256796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.257231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.257657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.308024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f224d-9622-41a2-8f1f-960c25bbd9d8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.332298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:11.343267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.345131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.369078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.369880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.370173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.370454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.391730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.393253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:11.436676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:11.438009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:11.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:11.444847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:11.450424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.471372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 354/692] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:57:11.478332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.568323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:11.672785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993c46f-433b-4f33-955b-0c297bcc2715 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.680517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:11.698485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c281e7-4eff-4b67-9bb4-29750ddfb9db None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:11.743437 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.744116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.745372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.745657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.746501 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:11.746810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.748249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.770207 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 339/693] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:11.778677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.789740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:11.790555 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:11.790837 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:11.791113 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:11.808383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:11.809892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:11.850288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:11.851319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:11.852305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:11.852305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:11.852305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:11.852305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:11.856422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 355/694] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:11.859631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:11.991716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4a617-790a-48c8-92a6-a5ed40d7f117 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.014810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:12.039545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13343cdd-b85d-46a7-98af-9a96167d2df6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.062527 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.064659 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.064949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.065231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.122585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.123176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.123460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.123738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.124496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:12.124814 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.126306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:12.145463 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 356/695] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:12.160326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:12.171544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:12.189628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.191210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:12.377835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3985f3e-61d7-4891-8059-2d0e2d8781fe None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.395124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 340/696] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:12.401740 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:12.418608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:12.456776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.457596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.457886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.458167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.474051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.476109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-6cf1dc8f-5951-4a2c-ac9c-e06169311457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:12.495591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:12.496733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:12.498207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 357/697] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:12.498759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839488-25b7-4b73-af4f-975257d77fbb None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.499051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839488-25b7-4b73-af4f-975257d77fbb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.499337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9839488-25b7-4b73-af4f-975257d77fbb None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.504928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 358/698] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:12.504928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839488-25b7-4b73-af4f-975257d77fbb None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:12.571514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a996dd-ddcc-44aa-915c-159f2863ae53 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.578204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:12.614098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.615008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.615296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.615599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.621375 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9839488-25b7-4b73-af4f-975257d77fbb None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.626470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.627988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-fd6b17db-4aa7-445d-b2e1-22d5e564a803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:12.649575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:12.651280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:12.653802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:12.653802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:12.653802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:12.653802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:12.656245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 341/699] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:12.657649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd49e-6a2a-4c71-bfc3-2f4b485d475a None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.657938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd49e-6a2a-4c71-bfc3-2f4b485d475a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.658223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fd49e-6a2a-4c71-bfc3-2f4b485d475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.679957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-fe5fd49e-6a2a-4c71-bfc3-2f4b485d475a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:12.681252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-fe5fd49e-6a2a-4c71-bfc3-2f4b485d475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:12.748444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:12.776592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 342/700] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:12.778073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbaff-a9fe-47b8-998f-3d4e53f82095 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.779470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbaff-a9fe-47b8-998f-3d4e53f82095 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.779750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fbaff-a9fe-47b8-998f-3d4e53f82095 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.785605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 343/701] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:12.786435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fbaff-a9fe-47b8-998f-3d4e53f82095 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:12.795830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.797226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.797794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.798541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.799596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:12.800135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.801901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41320754-a0d4-41e8-8905-9cf732e35a9b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:12.815834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 359/702] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:12.816755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.817041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.817323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.836338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:12.837586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:12.922604 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fbaff-a9fe-47b8-998f-3d4e53f82095 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:12.936444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 360/703] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:12.936444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:12.980336 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:12.981037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:12.981321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:12.981598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:12.982338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:12.982644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:12.984016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.000608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 344/704] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.000608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.008968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.059222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.063836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a761b1-be17-4b68-b5ed-d1cc531402e8 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.105108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.105939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.106227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.106527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.107176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.108593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.122231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5f677-fde2-4a06-b4aa-dcd864694968 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:13.133034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:13.134347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:13.135277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 361/705] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:13.136196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.163982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.164669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.164960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.165239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.166064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=af1926a9c0e146219e85e0021a489b59) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:13.166381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.167812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.174536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 345/706] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:12 2022] GET /identity/v3/role_assignments?user.id=af1926a9c0e146219e85e0021a489b59&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.175329 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.254335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f696d-cf1c-4b64-b5f3-52c317b30b04 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.275720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.298699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.299660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.299919 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.300048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.300230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2a920-bf20-42cf-8145-5f4b5b7c084f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.312496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.314110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.320878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-ec8cb4b9-d566-4b87-81b1-5a2fe4200d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:13.347537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:13.348847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:13.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:13.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:13.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:13.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:13.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 362/707] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:13.350609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894e0a-4b8a-4043-820f-1cb73d9ad2d6 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.350899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894e0a-4b8a-4043-820f-1cb73d9ad2d6 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.351178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97894e0a-4b8a-4043-820f-1cb73d9ad2d6 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.354413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 363/708] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:13.354767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97894e0a-4b8a-4043-820f-1cb73d9ad2d6 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.359673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.360538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.360831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.361108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.372539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.373976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:13.391713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:13.392863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:13.393878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 346/709] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:13.394427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.495107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97894e0a-4b8a-4043-820f-1cb73d9ad2d6 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.519138 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca64502-7a04-453f-826d-9cf194097bed None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.550244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.551043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.551402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.551751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.552583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:13.553316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.554706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a490e8-2cc7-4815-bc73-711cb192dbd2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.558280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.559389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.559739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.564830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 364/710] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.564997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:13.564997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.565183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.565183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.565183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.565438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.577025 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 347/711] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.578040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.587421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:13.588667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:13.616595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.620719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.677019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 365/712] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:13.677822 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.711394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44078189-d4a7-4000-a58f-e7493d54ef06 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.752757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.753352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.753790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.754081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.754883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:13.755198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.757389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.775568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 348/713] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.775568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.802251 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16877d8-3f62-436c-96e4-0a928fa7ff74 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.843136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.843896 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.844186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.844494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.845210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:13.845517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.846904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:13.860456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 366/714] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:13.860456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:13.904888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.931671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:13.941308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd67a2-31f7-4b9e-b21e-c1c645fc8319 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:13.984905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:13.986018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:13.986565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:13.987008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:13.987907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:13.989903 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:14.011352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472bf7-73e3-490f-bb4f-20f24df621e9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:14.017730 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 349/715] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:57:13 2022] POST /identity/v3/users => generated 254 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:14.018938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:14.106910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:14.107810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:14.108302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:14.108624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] PATH_INFO: `/v3/services/network` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:14.109571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:14.111129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:14.127126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:14.128326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:14.129318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 367/716] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:13 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:14.137557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:14.185873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2f8b2-634e-43f2-ae03-d52fd5e7052b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:14.271185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:14.272469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:14.272753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:14.273274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:14.274144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:14.274480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:14.276857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:14.284287 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 350/717] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:14 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:14.319231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:14.333152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec2ccf-c7a9-4fb0-9819-5ded16f9f8b9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:14.375502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:14.376666 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:14.377225 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:14.377739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:14.378709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] RBAC: Adding query filter params (name=network) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:14.379243 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:14.380937 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d744b-8852-4c9c-ae04-e6c2611ef437 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:14.387385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 368/718] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:14.393272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:14.418432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:14.531159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cebaa-c396-4954-b3df-ce37839d4f56 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:14.571948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:14.573002 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:14.573517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:14.573983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:14.574900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] RBAC: Adding query filter params (type=network) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:14.575406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:14.577009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5455f8e-779e-4272-9372-26fc80b84234 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:14.585443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 351/719] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:15.968847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80136-83ac-4ecf-b4f6-fda9b10978da None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:15.969411 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80136-83ac-4ecf-b4f6-fda9b10978da None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:15.969411 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa80136-83ac-4ecf-b4f6-fda9b10978da None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:15.971050 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 369/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:15.977140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:15.977600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:15.977685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.001137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:16.002376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:16.091538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 352/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:57:16.131722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa80136-83ac-4ecf-b4f6-fda9b10978da None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:16.277848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa80136-83ac-4ecf-b4f6-fda9b10978da None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:16.324999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:16.325821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:16.326258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:16.326543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.344441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:16.352984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:16.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:16.354968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:16.360217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 370/722] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:16.367460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:16.388525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:16.530314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28dcc5-bcbe-4086-8a25-d5a0fcb6d859 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:16.586516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:16.587192 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:16.587475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:16.587750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.588530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:16.588839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:16.590206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:16.599496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 353/723] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:16.605117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:16.648802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:16.778113 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaccd72-e728-4bd2-ad97-532e5093c2b2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:16.827857 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:16.829091 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:16.829604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:16.830108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.848455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:16.850918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-09a538e9-514a-4e46-a21d-d228a212976b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:16.886589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:16.887647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:16.888670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:16.888670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:16.888670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:16.888670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:16.890824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 371/724] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:16.901599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:16.938918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebde-a639-4b61-bde4-fc86cc4c1b1e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:16.939584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebde-a639-4b61-bde4-fc86cc4c1b1e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:16.940138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebde-a639-4b61-bde4-fc86cc4c1b1e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.943051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 372/725] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:16.961998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278683-ef6b-4494-bac8-b8660e72fa0c None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:16.962613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278683-ef6b-4494-bac8-b8660e72fa0c None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:16.963096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88278683-ef6b-4494-bac8-b8660e72fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:16.993675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-88278683-ef6b-4494-bac8-b8660e72fa0c None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:16.995984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:16.997151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-88278683-ef6b-4494-bac8-b8660e72fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:17.048921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd84e8c-313f-45c4-8684-51a6cc0ad921 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.073296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:17.079677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 373/726] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:17.082356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee68c4-91af-4c87-a238-22070e0e29f1 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.083044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee68c4-91af-4c87-a238-22070e0e29f1 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.083850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee68c4-91af-4c87-a238-22070e0e29f1 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.085975 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 374/727] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:17.087664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f97e58-2854-46a7-a9b2-03ccf18d245b None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.088161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f97e58-2854-46a7-a9b2-03ccf18d245b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.088726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f97e58-2854-46a7-a9b2-03ccf18d245b None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.090681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 375/728] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:17.092066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.092355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.092657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.100160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.101159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.101450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.101731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.102478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:17.102813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:17.104291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfaa20-6a93-4442-9acc-3a2d62329e37 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:17.111142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:17.112323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:17.118580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 354/729] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:17.119877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.120015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.120431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.137856 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:17.139140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:17.191850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 376/730] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:17.193107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.224889 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 355/731] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:17.226084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.299709 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:17.336112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913fc17-7144-461f-a943-de19a2f9b592 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.346827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb42b29-17be-4d99-a79c-5a51588960cf None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.367055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:17.377257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.378206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.378489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.378923 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.379974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:17.382061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:17.389599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.390815 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.391213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.391508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.408438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 377/732] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:17 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:17.408438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.408852 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:17.408852 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:17.426920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:17.428661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:17.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 356/733] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:17.430955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.573863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2e950-7ff6-4d58-89cc-2daf8523422c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.602996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99624b5-8c90-44c2-9ddd-e0390aab6e92 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.629053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.629975 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.630330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.630616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.635854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:17.647442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:17.652612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.653507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.653803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.654084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:17.655413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:17.656474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:17.657483 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 378/734] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:17.658279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.673768 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:17.682864 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:17.684428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:17.685412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:17.688503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 357/735] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:17.689380 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.720123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:17.788241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b41e0-6e82-43b7-8286-893de5cbc7ec None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.837766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.838838 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.839168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbb4d2-4604-4b25-84d4-e58d8d5fbc92 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:17.839297 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.839746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.841279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:17.841799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:17.843891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:17.878003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 379/736] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:17.878003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.898498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:17.899190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.899478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.899759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.900559 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:17.900875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:17.902746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f590-0141-4d56-b7d7-d7efc011fe1f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:17.911400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 358/737] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:17.913242 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a564d5-9bb3-4ff3-8069-799d17433fa7 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:17.913531 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a564d5-9bb3-4ff3-8069-799d17433fa7 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:17.913811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a564d5-9bb3-4ff3-8069-799d17433fa7 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:17.915227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 359/738] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:17.925382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a564d5-9bb3-4ff3-8069-799d17433fa7 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:17.981731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.007870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ede88-0fec-4145-9b09-ec9f51f92f19 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.010219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.051520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a564d5-9bb3-4ff3-8069-799d17433fa7 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.060182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.060901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.061182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.061462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.062228 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:18.062540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.063918 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.078481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 380/739] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:18.079420 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.097302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.098309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.098832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.099287 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.184542 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.186270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.196154 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d210d4e-31a7-434f-bb02-f2543d644c36 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.232324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.233304 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.233601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.233982 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.234867 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.247584 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.249096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-9bca8048-baa4-4a70-a3bb-3533d2953d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:18.266873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:18.267926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:18.268979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:18.268979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:18.268979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.268979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:18.272980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 381/740] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:18.273976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.274266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.274591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.292276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:18.293588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:18.309519 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 360/741] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:57:18.309519 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.310415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.377541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 382/742] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:57:18.378647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.445870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e218-7fcf-4ef1-8846-50f95a9eccb7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.488053 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.488877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.489397 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.489397 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.499730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb116-514d-431a-a353-ddd5030b0c23 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.512129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.512129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.513836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:18.534644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:18.535682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 361/743] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:18.537116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.552593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.554082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.554573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.554883 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.556083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:18.556083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.557486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.575773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 383/744] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:18.576715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.589524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.664598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97bfe90-3e13-493c-8566-0975363b6499 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.701788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac99302-7716-4e19-b831-a0c6acb42da1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.704051 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.704904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.705195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.705477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.718220 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.719694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:18.737686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:18.738737 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 362/745] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:18.739752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.746553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.747235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.747521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.747807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.748582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.749980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.768000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 384/746] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:18 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:18.768674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.821426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.869921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:18.907400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90bc63-7761-418c-bce3-c34785deea91 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.932132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6fea9-2b19-4174-af20-c4d64b6b92e6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:18.959511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.961004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.961682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.962326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.963339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:18.963656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.965760 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:18.977285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:18.978862 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:18.979139 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:18.979429 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:18.982325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 363/747] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:18.983879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:18.990195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:18.991784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:19.009892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:19.016449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 385/748] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:19.018998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.121738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337cb89-84b4-4a75-bbca-1df9a3098af8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.132984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.162887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.163538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.163828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.164110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.164877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:19.165212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.166812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.167533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5583f0b-fdfb-48cf-9161-51d42a56e1b8 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.185420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 364/749] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:19.186302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.214114 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.219770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.221167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.221784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.222317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.235180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.237277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:19.258758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:19.260082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:19.261719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:19.263816 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 386/750] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:19.265543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.326468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da641eaf-7031-4391-aef6-b0ea77431056 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.365111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.367186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.367483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.367764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.368648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:19.368968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.370441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.393084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 365/751] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:19.394038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.421101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3b711-15fe-4f94-8d70-f86bf9ff71eb None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.428929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.478491 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.480218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.480971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.481441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.492916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.494985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:19.514639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:19.515863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:19.517115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:19.520337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 387/752] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:19.523940 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.527546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a001f2-8741-469c-a594-a5666d8f1cc6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.575348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.579389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.581441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.581901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.582613 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.584123 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:19.585264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.587474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.617816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 366/753] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:19.618927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.646611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3ea08-323a-41dd-a420-a00a17a3f84d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.690527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.691503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.691954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.692381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.764541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.775712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.777560 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.782599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe58de-bab1-4e5a-8900-5de13bdbf74f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:19.822723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:19.823596 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:19.823980 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:19.824357 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:19.825426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:19.825697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:19.827233 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:19.829606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:19.842229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 367/754] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:19.843434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.903426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 388/755] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:57:19.904834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:19.957421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3d5b0-ef74-405a-aaff-4031f18957a2 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.001448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.002491 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.002845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.003192 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.017314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.018819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.022396 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23524c16-2a10-4f7b-bb29-28b200609559 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.025684 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-31705807-0141-48f7-a426-fb0237907df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:20.043072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:20.044305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:20.045759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:20.045759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:20.045759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:20.045759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:20.049313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 368/756] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:20.050688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6da62-ff29-4f07-93cd-665aaf4d0cc0 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.051111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6da62-ff29-4f07-93cd-665aaf4d0cc0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.051262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6da62-ff29-4f07-93cd-665aaf4d0cc0 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.052621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 369/757] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:20.054020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6da62-ff29-4f07-93cd-665aaf4d0cc0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.068024 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.068873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.069246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.069586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.070493 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=af1926a9c0e146219e85e0021a489b59) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:20.070874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.072394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.078892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 389/758] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:19 2022] GET /identity/v3/role_assignments?user.id=af1926a9c0e146219e85e0021a489b59&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:20.079780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.093482 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.186653 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6da62-ff29-4f07-93cd-665aaf4d0cc0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.212748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472fc88-f592-49a5-98e7-39c903adc3b6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.227629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.228506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.229040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.229460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.242218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.243922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.257132 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292023-5a66-4234-ad14-46036264d69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.258173 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292023-5a66-4234-ad14-46036264d69e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.258657 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292023-5a66-4234-ad14-46036264d69e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.259121 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292023-5a66-4234-ad14-46036264d69e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.260059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292023-5a66-4234-ad14-46036264d69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:20.260598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292023-5a66-4234-ad14-46036264d69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.262274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292023-5a66-4234-ad14-46036264d69e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-94d7d4d7-19d8-4084-8085-d4ffa00e53fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:20.264802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:20.265807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:20.266937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:20.266937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:20.266937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:20.266937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:20.266937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 370/759] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:20.268476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.268812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.269626 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.282083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 390/760] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:20.283452 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292023-5a66-4234-ad14-46036264d69e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.290399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:20.291713 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:20.345418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.370405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.372863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 371/761] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:20.374120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.411379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292023-5a66-4234-ad14-46036264d69e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.451762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.452621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.452910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.453192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.454806 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:20.455127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.456628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.472471 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 391/762] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:20.473710 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.502471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df535eb-edd6-40c2-a472-e56a6247819d None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.546532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.547669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.548357 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.548808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.557425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.559203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:20.574884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:20.576098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:20.577352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:20.579393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 372/763] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:20.580442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.628524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc3292-f8b8-4785-84f2-999057e3df5e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.632265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.665894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.680431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.681621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.682107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.682593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.683464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.685124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:20.703795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:20.705199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:20.706168 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:20.708479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 392/764] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:20.709614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.728495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef6a57-d01c-4c47-8744-a329887022e6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.780817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.782493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.782998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.783470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.794308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.796196 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:20.816515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:20.817595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:20.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:20.825827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 373/765] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:20.826890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb519d-7979-4a53-b658-1d918b65586b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:20.888137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:20.890305 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:20.890766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:20.891227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:20.892191 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:20.892721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:20.894438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:20.919914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 393/766] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:20.921423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:20.989542 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.994809 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:20.996237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908936-20b4-47d8-a2c8-4ff1183cccd6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.042694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60808391-a50f-4678-b867-2418de743925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.043899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808391-a50f-4678-b867-2418de743925 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.044366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808391-a50f-4678-b867-2418de743925 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.044829 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808391-a50f-4678-b867-2418de743925 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.045725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808391-a50f-4678-b867-2418de743925 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:21.046199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808391-a50f-4678-b867-2418de743925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.048825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808391-a50f-4678-b867-2418de743925 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.055165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 374/767] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.056436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60808391-a50f-4678-b867-2418de743925 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.076197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda5459-974c-4dac-abf8-a8c060dfcb40 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.117618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.118505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.118914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.119417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.120336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:21.120743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.122378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.141393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 394/768] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.141393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.197068 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60808391-a50f-4678-b867-2418de743925 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.242683 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.243903 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.244197 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.244583 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.246927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=482a2286f928483b8bce71f07fb4809f, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.248059 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.272439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 375/769] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:21 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:21.272439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.284600 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781dd3-e6a5-4b0c-b804-e3cc5aa226bd None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.299976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:21.309956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:21.344921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.345787 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.346509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.346909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.348447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:21.348897 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.351367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0df9ae-7ce7-4377-a4df-a112ac66634b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.363561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 395/770] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.363561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483a04-cffd-4663-b3d3-d9f46c3a2d11 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.363561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483a04-cffd-4663-b3d3-d9f46c3a2d11 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.370295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483a04-cffd-4663-b3d3-d9f46c3a2d11 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.373405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 396/771] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:21.373405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10483a04-cffd-4663-b3d3-d9f46c3a2d11 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.432776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c69c-75b2-41fd-b1d9-43a5f4f46aaa None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.479022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.480241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.480776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.481512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.483340 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=b37008507f684731b4687e5498da4ef5) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:21.484098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.485990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07c71f-3e2e-4bd6-a8a8-cf610af838c2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.493801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 376/772] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/role_assignments?user.id=b37008507f684731b4687e5498da4ef5&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.495601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.496124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.496646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.508269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10483a04-cffd-4663-b3d3-d9f46c3a2d11 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.529512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:21.531241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:21.553998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.554888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.555407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.555950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.556822 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] RBAC: Adding query filter params (interface=public, service_id=fb1f509a03a045838d2bf1e292d5542d, region_id=RegionOne) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:21.557144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.558561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.567927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 397/773] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/endpoints?service_id=fb1f509a03a045838d2bf1e292d5542d&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.574635 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.617519 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 377/774] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:21.626789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:21.644778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:21.666016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:21.705714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c1f0a-fffd-40ff-af5f-4e9f974c7d7f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.748364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.749317 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.749706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.750086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.750891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.752421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.760003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 398/775] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:21.772348 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df8cc6-ccf7-403d-90a4-47dbdf99b70b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:21.823423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:21.824572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:21.825017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:21.825472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] PATH_INFO: `/v3/services/network` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:21.826297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:21.827960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:21.853142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:21.854506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:21.855632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:21.858442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 378/776] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:21.864950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:22.032229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb7737-d49d-463f-b49f-446c6341c044 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:22.033922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:22.080295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:22.080961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.081202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.081490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.082218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] RBAC: Adding query filter params (name=network) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:22.082532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:22.083957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:22.089887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 399/777] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:22.096285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:22.208280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:22.232826 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f6f2d-abda-446f-87ba-c56ac7e461fd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:22.285455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:22.286335 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.286704 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.287078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.288631 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] RBAC: Adding query filter params (type=network) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:22.289022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:22.290480 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:22.301514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 379/778] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:22.308104 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:22.464321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d48ad-5589-47b5-91ca-ce35e3ff714a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:22.531830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:22.532620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.532988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.533343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.534223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] RBAC: Adding query filter params (interface=public, service_id=20fe5c31c6374bfa91930c9b63bf4480, region_id=RegionOne) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:22.534618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:22.536077 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e00cb-cef6-4954-a89c-7160b51ed792 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:22.544969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 400/779] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:22 2022] GET /identity/v3/endpoints?service_id=20fe5c31c6374bfa91930c9b63bf4480&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:22.550919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:22.735345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695078f-cbc2-4000-8e19-2d7938fabbce None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:22.759839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93eb6-85a2-4347-9b91-dec4c89c48a3 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.760217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93eb6-85a2-4347-9b91-dec4c89c48a3 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.760606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd93eb6-85a2-4347-9b91-dec4c89c48a3 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.762302 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 401/780] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:22.778989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.779299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.779698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.792004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:22.792717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:22.793009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:22.793787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:22.794489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:22.796201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:22.802863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 380/781] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:22 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:22.809873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:22.811260 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:22.869496 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:22.945250 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 402/782] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:22.988393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:23.000949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:23.116341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c60d9-bad6-401d-a5fc-29de17b753ed None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:23.153927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:23.154738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:23.155027 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:23.155300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:23.166497 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:23.174541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:23.175564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:23.176556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 381/783] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:23.184013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:23.346519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ed56a-fbdf-48cf-ae7a-c8052405feec None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:23.394613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:23.395738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:23.396255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:23.397637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:23.398858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:23.399449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:23.401224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:23.417120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 403/784] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:23.423409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:23.550051 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:23.602498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f3571-d652-4604-a6a1-4ee499643bf1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:23.655301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:23.656618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:23.657065 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:23.657525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:23.679621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:23.682503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:23.721277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:23.722339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:23.723873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:23.723873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:23.723873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:23.723873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:23.723873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 382/785] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:23.730969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:23.783260 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:23.908502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d37-5a7a-48ff-8aa3-085504408d4c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:23.964319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:23.965803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:23.966479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:23.967196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:23.969033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:23.969841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:23.972311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe54a10d-9daa-41c5-a66b-e5a9f5c939f1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:23.991884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 404/786] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:23.997508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.030082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b120-95a4-4904-9c8d-fe7f1e5edd91 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.030435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b120-95a4-4904-9c8d-fe7f1e5edd91 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.030654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a16b120-95a4-4904-9c8d-fe7f1e5edd91 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.032321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 405/787] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:24.050265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.050558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.050998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.077639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:24.079271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:24.207978 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94e05-c4ba-4760-912b-00cad045aa71 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.241711 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 406/788] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:24.252800 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:24.267493 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.284250 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.285112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.285402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.285686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] PATH_INFO: `/v3/users/swift` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.288741 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:24.304231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:24.305738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:24.331775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:24.333012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:24.334779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 383/789] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:24.339510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.466684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d3ae7-0775-4f9d-b5d7-2da4d12a6dbc None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.523844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.525137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.525637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.526108 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.528010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93860513-734e-4d0c-8f47-72fcda161e8e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.539752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a6aea88d-aafe-4f38-b82e-d21039c28ba1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:24.549369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:24.550595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:24.551639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:24.555065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 407/790] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:24.556869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b4bb0-23b9-4f7a-be05-4254f09e058e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.557510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b4bb0-23b9-4f7a-be05-4254f09e058e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.558018 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b4bb0-23b9-4f7a-be05-4254f09e058e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.560133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 408/791] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:24.561109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b4bb0-23b9-4f7a-be05-4254f09e058e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.571600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.572297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.572632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.572917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.573833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:24.574187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:24.575675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095c7b7-562d-47cc-ab73-88c5ad1b8aa2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:24.602673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 384/792] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:24.602673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.602673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.602673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.618122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:24.619394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:24.658707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:24.675853 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:24.701358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b4bb0-23b9-4f7a-be05-4254f09e058e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.712685 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 385/793] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:24.714037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.743810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.744538 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.744942 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.745346 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.746363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:24.746807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:24.748517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:24.760850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 409/794] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:24.761840 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.853968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececb95a-ed18-4a7e-b351-b4855a9a7596 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.889986 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a2793f-85ed-4bd8-bf2a-f6ef02a4506d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:24.907188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.908379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.908927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.909426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.921326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:24.922862 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:24.929497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:24.930300 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.930591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.930873 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.932843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:24.941957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:24.944630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:24.945693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 386/795] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:24.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a82f332b-ed46-4fc2-8b5e-6a33b2a56370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:24.952363 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:24.953350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 410/796] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:24.954353 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80fed-9ed0-4374-a3e9-c316d69204fc None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:24.955273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80fed-9ed0-4374-a3e9-c316d69204fc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:24.955273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d80fed-9ed0-4374-a3e9-c316d69204fc None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:24.961264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 411/797] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:24.961264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d80fed-9ed0-4374-a3e9-c316d69204fc None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:24.982396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.075572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d80fed-9ed0-4374-a3e9-c316d69204fc None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.083301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52a7dc-32e6-489c-826d-4d9f9126a7dd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.111442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.112146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.112441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.112726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.113477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:25.113787 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.115721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.127958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.129047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.129187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 412/798] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:25.129595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.129904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.130210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.144524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.146238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-82f8ab00-e9f6-4bed-bce2-eaeed2acecbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:25.174749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:25.175868 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:25.176907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:25.176907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:25.176907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:25.176907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:25.180630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 387/799] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:25.182624 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.182922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.183207 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.198533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.201124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:25.202407 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:25.252925 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696c0a5-501d-4a4f-a63d-c1cb1c2b524c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.286347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 388/800] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:25.292486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.293142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.293434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.293714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.294444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.294561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:25.294794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.296175 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.296961 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.312437 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 413/801] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:25.313738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.422697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08f3f6-3981-48d4-88a1-3de6cee1832b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.434793 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09c023-9508-46ec-96fa-4a8a20ad37f3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.452306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.462143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.462957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.463249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.463529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.471556 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.473087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.475530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.476291 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.476836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.477122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.478299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:25.478615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.480037 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:25.490188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:25.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 389/802] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:25.494672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.503561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 414/803] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:25.503561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.574399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.621469 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1ed1-f3a3-4216-bcc6-5c17dd5c8bf8 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.627797 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b53976-9288-4919-807d-edb2e9a0bdd0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.665812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.666072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.666633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.666761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.666932 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.667040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.667220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.667329 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.667951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.669337 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:25.682808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:25.684043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:57:25.685086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:25.688733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.688895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 390/804] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:25.689521 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.690724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:25.711415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:25.712747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:25.713926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:25.713926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:25.713926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:25.713926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:25.715525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 415/805] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:25.716497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.754459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.822974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e822ea-421e-4fb4-af24-093c8051da0c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.835265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbce22c-a985-41d8-b3a7-0eef57cfa1cc None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:25.849943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:25.859850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.860560 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.860847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.861124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.861915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:25.862226 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.863674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.878004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:25.878827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:25.879115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:25.879389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:25.886616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 391/806] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:25.887726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:25.888096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:25.889637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:25.908048 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:25.909599 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 416/807] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:25.910675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.002194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55fd21f-c608-4f9a-b88c-3760da6dd3ed None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.002880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.016537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb49d3-8945-4478-9159-27f8676a5758 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.036554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.037830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.038120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.038401 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.039103 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:26.039406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.040799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d7475e-0a57-4f4e-bd43-19bb92e36289 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.045927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 392/808] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:26.047132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828227-3597-4bc8-8b23-785d321d7f33 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.047418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828227-3597-4bc8-8b23-785d321d7f33 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.047693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23828227-3597-4bc8-8b23-785d321d7f33 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.049516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 393/809] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:26.050096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23828227-3597-4bc8-8b23-785d321d7f33 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.058752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.059858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.060160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.060480 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.061227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:26.061540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.062928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.075394 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 417/810] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:26.076226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.102474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.157245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23828227-3597-4bc8-8b23-785d321d7f33 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.187683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289cba93-d4f5-4d9a-9414-63e1aeb44597 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.192670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.193718 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.194025 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.194282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/af1926a9c0e146219e85e0021a489b59/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.196343 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=af1926a9c0e146219e85e0021a489b59, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.197959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90c8b2-4a7e-4e72-af65-7db5d4fa2191 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.214436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 394/811] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:26 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/af1926a9c0e146219e85e0021a489b59/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:26.215742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.216040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.216327 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.235612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:26.236912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:26.238431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.239150 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.239441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.239727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.241529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:26.241966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.243475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.270120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 418/812] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:26.270120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.280996 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.342218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 395/813] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:26.343459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.407237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737827c-7a8a-4e4c-b789-b6416737189e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.419001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.453194 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.453882 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.454164 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.454439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.455151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:26.455456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.456884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.464165 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 419/814] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:26.466211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.493646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69033810-f4d9-4601-9520-b9bfb109a905 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.535894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.537715 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.538095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.538400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.550326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.552971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.554725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:26.571252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:26.572516 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 396/815] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:26.577940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.603289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc1f16-c0ea-4405-9c4b-dce3595ba16d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.643189 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.644101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.644389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.644698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.656445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.658001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:26.678880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:26.679949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:26.681008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:26.681008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:26.681008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:26.681008 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:26.685126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 420/816] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:26.686028 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.692866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.696379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7527a-ae27-47d4-ad1c-adebadad64e2 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.734814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.736036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.736036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.736291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] PATH_INFO: `/v3/services/network` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.736828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.738193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:26.750937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:26.752106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:26.753411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 397/817] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:26.754322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.811965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5d82e-16df-4708-a25e-ee90b8b5c826 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.839420 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:26.853161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.853828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.854117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.854396 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.855151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.857448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.884844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9fdf0-a5ae-4a59-a464-d444dfc5914f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:26.896884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 421/818] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:26.896884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:26.942824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:26.944325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:26.944697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:26.945091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:26.945966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:26.946285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:26.947757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:26.975193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 398/819] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:26.976301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.008559 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.045852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bc7a3-8ba2-4cba-8915-6240bcc58147 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.097076 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.098179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.098567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.098943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.100858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:27.101446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.103349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.110158 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf0ed-86c4-49f3-bc16-1619d6da2cec None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.122307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 422/820] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:27.123744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.155528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.156308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.156630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.156909 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.157626 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] RBAC: Adding query filter params (name=network) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:27.157937 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.159455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.166666 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 399/821] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:27.167484 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.178446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.261991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662813ca-5147-46dd-8b72-9eddbf95533e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.277291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.297862 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d65f21-e2be-46b3-a4c2-a07d66e7d4ed None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.304985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.305656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.305943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.306216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.336890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.338182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.338571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.346347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.348098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:27.363528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:27.364824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:27.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:27.367563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 400/822] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:27.372280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.387160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.388789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.438425 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.486128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f425bb3-b652-4882-ad26-a8ffb37b2b04 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.509723 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 423/823] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:57:27.511049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.536096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.537253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.537253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.537456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.538147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] RBAC: Adding query filter params (type=network) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:27.538481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.539987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.545614 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 401/824] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:27.546504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.564845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.625051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223524c5-1e6b-41bf-8701-40af88e127ef None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.657020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d81cf-21d4-447b-98ba-ae6bebddff39 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.663303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.664300 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.664782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.665250 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.666223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:27.666691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.668325 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.676473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.687218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 424/825] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:27.688169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.700912 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.702152 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.702758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.703297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.716799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.718831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:27.738570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:27.739638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:27.740778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:27.740778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:27.740778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:27.740778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:27.743403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 402/826] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:27.744792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.800448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43b9d5-c6ec-4251-a94f-5f993e8d67f3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.820796 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.838517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.839196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.839542 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.839826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.840630 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.842032 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.861139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726b6b-7a53-45ad-89df-9fb8108ef3de None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:27.864104 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 425/827] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:27.866996 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:27.917876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:27.919391 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:27.920011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:27.920570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/b37008507f684731b4687e5498da4ef5/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:27.938569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=b37008507f684731b4687e5498da4ef5, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:27.940492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:27.949303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:27.958166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 403/828] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:27 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/b37008507f684731b4687e5498da4ef5/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:28.038944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961c0e9-a76a-4d1f-b52a-d8db40a61085 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:28.093187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:28.093893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.094183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.094464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.095229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:28.095545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:28.097039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:28.112218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 426/829] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:28.118270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:28.266322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308792f4-f226-4dbc-9a81-742c8271c008 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:28.269577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:28.311572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:28.312393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.312708 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.312990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.321054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:28.322523 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:28.346332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:28.347374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:28.348361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 404/830] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:28.354662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:28.471227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab931d-708a-4bdd-bc00-716995eb37c4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:28.492631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:28.516199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:28.516967 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.517193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.517474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.518811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:28.518811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:28.524439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42922-4b65-4225-a075-b2ec2c25b2ae None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:28.540460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 427/831] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:28.548729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:28.714295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ba345-d170-417d-981b-56fed9855346 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:28.751981 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:28.752700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.752986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.753263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.754115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:28.754429 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:28.756570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3779f12-1635-4cc0-928d-650eda977b89 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:28.766090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 405/832] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:28 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:28.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5a1-b8d2-43a0-802a-4967c1e20cbe None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.838676 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5a1-b8d2-43a0-802a-4967c1e20cbe None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.838961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c5a1-b8d2-43a0-802a-4967c1e20cbe None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.840765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 428/833] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:28.846830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:28.847136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:28.847427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:28.871085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:28.872310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:28.900435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:28.959869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 406/834] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:28.980328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c5a1-b8d2-43a0-802a-4967c1e20cbe None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.104691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c5a1-b8d2-43a0-802a-4967c1e20cbe None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.141561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.142373 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.142838 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.143126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.154673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:29.163378 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:29.164521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:29.165549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 429/835] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:29.173637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.291384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e912e594-6c40-4957-bf12-27fa4fef807e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.331673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.331673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.332213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.332213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.332790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:29.333098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:29.334474 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8be8e-10a6-4f5f-af10-769e605745b7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:29.344010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 407/836] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:29.349250 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.393839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:29.472099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a94de0-8bd9-4662-b3bb-17cc7a307c75 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.493582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eaa7-b4b2-447a-be0c-72e32f78f8df None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.493879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eaa7-b4b2-447a-be0c-72e32f78f8df None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.494160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eaa7-b4b2-447a-be0c-72e32f78f8df None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.497109 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 408/837] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:29.501246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.501533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.501813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.510883 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.511807 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.512102 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.512387 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.524648 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:29.526241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:29.533276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:29.534886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:29.546890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:29.547926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:29.549881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:29.549881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:29.549881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:29.549881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:29.549881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 430/838] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:29.554188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.557649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:29.627019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 409/839] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:29.659712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.684281 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:29.708347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b9b57b-316b-4f08-a856-d5ecc1306897 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.758774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.759480 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.759769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.760057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.761007 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:29.761322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:29.762712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:29.776478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 431/840] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:29.778263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b9a4c-4eb4-4848-afa9-accc8af8d772 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.782393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.817863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.819880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.820324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.820778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.821605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:29.823182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:29.836358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:29.837538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:29.838524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:29.846298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 410/841] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:29.848018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:29.866397 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:29.909924 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc242a5-1048-41b5-894f-75c3e43b10c1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.952165 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:29.953052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:29.953345 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:29.953628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] PATH_INFO: `/v3/users/swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:29.961885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640fe24-bbad-4f0b-9f2d-f67d569c4c58 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:29.962780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:29.964385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:29.982593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:29.983695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:57:29.984761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:29.988699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 432/842] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:29.989698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.004307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.004866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:30.006213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.006810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.007410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.008354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:30.008896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:30.010602 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:30.020786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 411/843] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:30.022996 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.117148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:30.131684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216484d-dce0-46a9-95b9-9b6d4af69f2a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.142091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583d954-9598-4b7a-89fc-a9f597cfc900 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.172715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.173755 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.173964 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.174229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.175068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:30.175376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:30.176960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:30.183493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.184511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.185007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.185506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.186428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:30.186961 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:30.188578 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:30.199861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 412/845] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:30.203674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 433/845] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 19:57:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:30.205070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.330120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c609c-8e38-420b-9172-b4ca4a4a10a7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.371126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.372094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.372589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.373033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.388050 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:30.460069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:30.461983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48b538-4a41-46f7-86f8-bbcf4057f576 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:30.586051 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 413/846] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:57:30.591864 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.596579 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf278f-3df6-4907-8c16-80621952d480 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.596876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf278f-3df6-4907-8c16-80621952d480 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.597165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf278f-3df6-4907-8c16-80621952d480 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.598806 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 414/847] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:30.604970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.605270 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.605545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.631564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:30.633007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:30.680807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:30.697005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:30.742981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed28a2-e57b-42d5-8e19-bb601b0a1324 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.753538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 415/848] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:30.783895 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.793242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.794444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.795039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.795571 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.808283 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:30.809828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:30.831543 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:30.832611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:30.833604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:30.833604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:30.833604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:30.833604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:30.835074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 434/849] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:30.843158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.908733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da6e3f-cc01-485e-adee-d4dda16d9b27 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.953126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:30.953936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:30.954224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:30.954514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:30.966649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:30.970095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66596c-fe0a-4817-9882-5593cf795915 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:30.975076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:30.976124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:30.977260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 416/850] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:30.982406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:30.992524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.013813 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.014902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.015412 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.015941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.017030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:31.017830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.019431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.025852 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.037234 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 435/851] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:31.048460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.115119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4dd55-71c1-4681-bdb4-2fc5f79e1be0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.160561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.162135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.162898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.163471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.164654 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:31.165155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.167338 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.175798 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588019a0-39c5-4821-bfce-78709d29268f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.180505 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 417/852] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 19:57:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:31.184816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.225888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.226861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.227151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.227428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.234898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.236380 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-3f08a923-01c3-4843-8a26-6488811b5c64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:31.252064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:31.253286 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:31.254278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 436/853] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:31.254823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f859e7-0be5-4062-a274-383a054724b6 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.254823 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f859e7-0be5-4062-a274-383a054724b6 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.255612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f859e7-0be5-4062-a274-383a054724b6 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.256745 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 437/854] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:31.262022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f859e7-0be5-4062-a274-383a054724b6 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.306118 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.311036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.316771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00711bb4-1a94-424d-bd4e-4b58396c04b8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.365201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.366495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.366849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.367135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.378814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.380463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.391660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f859e7-0be5-4062-a274-383a054724b6 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f78711ae-d9da-49ec-bc83-09e0513b19e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:31.398436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:31.399471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 418/855] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.401524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.401982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.426462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:31.429081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:31.435443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.436122 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.436439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.437512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:31.437825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.439270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.472812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 438/856] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:31.472812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.547372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 419/857] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:31.548573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.594145 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.602231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e823ce-c165-4a24-8da7-1efda81c503d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.639741 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.647558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.648241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.648674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.648955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.653642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:31.653642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.653642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.666906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96051538-a60b-4d68-b04c-45df4f8d0acf None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.672349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 439/858] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:31.672349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.714020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.714597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.714882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.715157 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.715990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=af1926a9c0e146219e85e0021a489b59) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:31.716302 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.717857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.726191 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 420/859] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/role_assignments?user.id=af1926a9c0e146219e85e0021a489b59&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:31.726920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.814841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daeba-f385-4b22-a2d2-68cd8b0a13d7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.862202 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1dc79-d1fc-4a9f-b754-4c454467f990 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:31.864129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.866913 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.867481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.867953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.868894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.871203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:31.890344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:31.891208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:31.891969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:31.899463 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 440/860] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:31.901279 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:31.912820 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:31.913667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:31.913948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:31.914218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:31.923569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:31.925171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:31.929068 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.939444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:31.945077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:31.945851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:31.946667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 421/861] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:31.952688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.058533 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a623-756e-42ab-a489-03c7847cb2ee None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.095481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.096330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.096738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.097175 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.097962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:32.098373 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:32.099943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:32.104050 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec1eec-af99-4c52-9cd6-afe0b3c685a3 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.105946 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 441/862] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:32.112642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.149841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.151451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.151885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.152308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.153995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:32.154456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:32.156608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:32.187292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 422/863] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:32.188878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.234007 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:32.258366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8838c-dbf1-4183-a81d-f7c1de3ecc3f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.263325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:32.306406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.307287 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.307650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.308306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.309137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:32.309419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:32.310886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:32.311658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88d60f-baca-4840-b0f0-d6f2c0e72c55 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.323908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 442/864] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:32.357742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.358542 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.358896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.359332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.449281 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:32.451386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75dbae-13a6-4849-bf03-0ff14ecf25d4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:32.532871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:32.579914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 423/865] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:57:32.586372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.689482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67969b-4c4f-4b20-9d71-92d5241e0d95 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.690136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67969b-4c4f-4b20-9d71-92d5241e0d95 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.690613 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67969b-4c4f-4b20-9d71-92d5241e0d95 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.692514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 424/866] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:32.702236 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.702523 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.702805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.713821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d79bb-3e1a-44e4-bcd5-5c447117cc59 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.722984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:32.724380 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:32.760520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.761674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.762141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.762589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.778133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:32.780621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:32.792169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:32.806385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:32.807185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:32.809321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:32.809321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:32.809321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:32.809321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:32.809321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 443/868] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:32.810173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 425/868] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:32.812428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.830928 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:32.935412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:32.950860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554bbff-8450-42e7-af72-51ce04529615 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.952280 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e48ce-3171-42a0-ada1-b5ef10439199 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:32.995609 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:32.996811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:32.997355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:32.997860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:32.998851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:33.000508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:33.001285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:33.002092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:33.002439 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:33.002787 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:33.003653 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:33.004470 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:33.007339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:33.027477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 444/870] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:33.027749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 426/870] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:32 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:33.029808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:33.103967 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:33.152103 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7528f-548c-4728-a064-6cc804a7163f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:33.200005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:33.200850 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:33.201131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:33.201405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:33.209267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:33.210888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:33.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:33.228023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:33.229431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 445/871] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:33.235817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:33.359446 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5270-94fc-4fef-9540-c1ee56fc434f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:33.397969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:33.398918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:33.399371 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:33.399811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:33.400818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:33.401299 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:33.402893 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8702d2-4e63-42e8-8cea-67ee1d7b9888 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:33.417484 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 427/872] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:33.423933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:33.535897 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4c007-d77f-4dd4-b7cf-28d0eb9163d6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:33.556775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:33.575725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:33.576437 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:33.576716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:33.576992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:33.577856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=b37008507f684731b4687e5498da4ef5) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:33.578166 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:33.579665 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5df9c2-834d-4e03-84f7-cb95ef1b6ab5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:33.587655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 446/873] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:33 2022] GET /identity/v3/role_assignments?user.id=b37008507f684731b4687e5498da4ef5&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:34.154421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72eac-2b18-4597-9e61-29e8c783a596 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.154927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72eac-2b18-4597-9e61-29e8c783a596 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.155027 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a72eac-2b18-4597-9e61-29e8c783a596 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.156608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 428/874] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:34.162827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.163332 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.163403 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.182706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:34.184257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:34.261009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 447/875] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:34.282869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a72eac-2b18-4597-9e61-29e8c783a596 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:34.294321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:34.402465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a72eac-2b18-4597-9e61-29e8c783a596 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:34.441765 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:34.442507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.442795 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.443611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.444493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:34.446024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:34.459923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:34.461078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:34.461793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 429/876] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:34.467354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:34.592916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97000ca3-3571-452e-89e7-59f5d408126c None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:34.605301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:34.629679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:34.630409 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.631581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.631964 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.632708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:34.633047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:34.634535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a165-dfad-4e93-8027-0f07f2b9ed42 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:34.640462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 448/877] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:34.646296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:34.722381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da06ba-cfeb-4339-845d-86dd21ebb957 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.722661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da06ba-cfeb-4339-845d-86dd21ebb957 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.722860 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da06ba-cfeb-4339-845d-86dd21ebb957 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.724660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 449/878] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:34.729947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc16e-3e28-4714-8f9a-d933080356cd None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.730342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc16e-3e28-4714-8f9a-d933080356cd None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.731025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cc16e-3e28-4714-8f9a-d933080356cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.733137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:34.755356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-267cc16e-3e28-4714-8f9a-d933080356cd None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:34.757776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-267cc16e-3e28-4714-8f9a-d933080356cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:34.771046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54cd513-ae49-4249-8acf-47e2a8ce7f30 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:34.810579 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:34.811254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:34.811543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:34.811821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:34.812691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:34.813003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:34.814508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:34.820751 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 430/879] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:34.851183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 450/880] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:34.875569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:35.004236 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f202694-c7a2-4ad3-a106-280c30dab75c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:35.042668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:35.043346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:35.043633 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:35.043908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:35.044674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:35.046054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7c46-c617-4ade-9c67-f2b0b9961b8d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:35.050411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:35.066876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 431/881] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:34 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:35.570736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b72a1f-7405-4022-ad60-5a6655690caa None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:35.571405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b72a1f-7405-4022-ad60-5a6655690caa None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:35.571405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b72a1f-7405-4022-ad60-5a6655690caa None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:35.573215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 451/882] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:35.579406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:35.580022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:35.580022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:35.598113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:35.599372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:35.688785 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 432/883] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:35.712118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b72a1f-7405-4022-ad60-5a6655690caa None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:35.823733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b72a1f-7405-4022-ad60-5a6655690caa None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:35.826119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:35.861672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:35.862477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:35.862761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:35.863035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:35.863696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:35.865629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:35.878685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:35.880839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:35.882464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 452/884] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:35.893704 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.005457 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10541d5f-e183-4490-bfac-d033ea31494e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.034146 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:36.047907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.048884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.049175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.049451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.050173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:36.050484 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.051863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f8191b-33da-43ba-abb1-f0266f9a2ab8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.057915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 433/885] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:36.063410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.081255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeb49e-dd6f-4a22-9a3a-b26f1db44dc2 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.081255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeb49e-dd6f-4a22-9a3a-b26f1db44dc2 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.081255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abeb49e-dd6f-4a22-9a3a-b26f1db44dc2 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.082841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 434/886] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:36.091317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.091317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.091317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.108075 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:36.109420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:36.190100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab9fb7-08d9-41f4-a799-a3da447f7d01 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.212435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 435/887] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:36.239729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.247615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.248351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.248672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.248949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.249701 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:36.250015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.251609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.263127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 453/888] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:36.265740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.312878 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:36.385394 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1835bd-e5b4-4e55-b314-4ca5377e0076 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.397181 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36258972-35a3-4550-89ea-9daec9e076ff None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.434010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:36.438537 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.439209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.439494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.439770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.440594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] RBAC: Adding query filter params (interface=public, service_id=1b34990dd2bf402eb82ba6fabe1ad08b, region_id=RegionOne) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:36.440910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.442071 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.442290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.443823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.444330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.444821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.452562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 454/889] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/endpoints?service_id=1b34990dd2bf402eb82ba6fabe1ad08b&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:36.455461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.465411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:36.478001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:36.478875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:36.484674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 436/890] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:36.486800 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.586587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:36.594487 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d03d74-8335-4cf3-85eb-0f9475ce2ed5 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.611777 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb67e0-c6ca-4fae-a662-03ab04cbb401 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.640589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.641433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.641726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.642006 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.643052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.644788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.654710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.655746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.656232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.656746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.657748 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:36.658265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.659882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.660860 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 455/891] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:57:36.676582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 437/892] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:36.679364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.813002 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be745764-9793-4b31-949b-e05f59921e7d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:36.856490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:36.857700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:36.858221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:36.858724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:36.872734 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:36.874715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:36.897552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:36.898432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:36.899211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:36.899211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:36.899211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:36.899211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:36.904461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 456/893] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:36.907161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:36.960622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.029897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4fcb48-0cdd-4576-ba82-ac2734951a4e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.072535 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.073519 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.073953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.074374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.075283 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:37.075742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.077363 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b9ea6-ec83-454c-8413-3ff502056a77 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.092621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 438/894] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:37.098945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.117203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.214231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5cd36e-5b86-44b1-8933-d9ed59c1a655 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.251851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.252958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.253382 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.253797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.262642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.264360 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.270755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc9b1f-467e-4a47-acdd-6ace45f6ed23 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.271073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc9b1f-467e-4a47-acdd-6ace45f6ed23 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.272139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc9b1f-467e-4a47-acdd-6ace45f6ed23 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.273712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 439/895] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:37.281666 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.282318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.282803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:37.286193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:37.287659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:37.288957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:37.292539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 457/896] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:37.295107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.311249 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:37.312823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:37.380426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.406170 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 440/897] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:37.420948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c295f57-89de-471e-9c01-affe0f473eda None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.430494 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.466755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.470535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.471309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.471729 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.472206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.473111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:37.473433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.475195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6692df92-4716-4294-87d5-4e1e26477927 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.493501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 458/898] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:37.495143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fed51-ab05-416e-95d4-089d4976e3f7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.495673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fed51-ab05-416e-95d4-089d4976e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.496145 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fed51-ab05-416e-95d4-089d4976e3f7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.497763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 459/899] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:37.501099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fed51-ab05-416e-95d4-089d4976e3f7 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.557687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9b42e-c804-448c-a58a-41c0c33b822e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.595921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.596658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.596953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.597236 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.598189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.599396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713f4a4-096d-46d3-a423-4d98fad71466 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.619137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 441/900] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:37 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:37.621317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.621816 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.622272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.641838 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:37.643410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:37.645035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fed51-ab05-416e-95d4-089d4976e3f7 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.679825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.690754 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.691539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.691899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.692242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.745487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 442/901] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:37.778062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.783675 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.785347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.804824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.910660 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a037f-f9be-474a-a7cb-ace5dea5d8f6 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:37.917507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 460/902] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:57:37.923602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:37.956874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:37.971282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:37.971282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:37.971282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:37.971282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:37.971680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:37.971680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:37.982988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:37.983865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:37.984727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 443/903] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:37.992147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.067791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3600cb-1168-47a7-ae03-ab8a0e739590 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.100917 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.112000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.113161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.113661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.114143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.118608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8c11c-650b-44a2-936f-cfe6cbec97d2 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.126849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.128886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:38.150296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:38.151155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:38.151915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:38.151915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:38.151915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:38.151915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:38.154260 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 461/904] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:38.161085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.166869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.167571 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.167860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.168144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.169224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.170623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.183253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 444/905] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:38.185397 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.257959 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.281797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d608a6-cf78-4717-98ea-30ec7b0d166e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.313823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911e3f3-b33f-483b-b168-d9bd8d01d801 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.320914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.321573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.321866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.322147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.322891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.323206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.324633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.343143 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 462/906] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:38.348509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.365258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.366200 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.366498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.366787 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.368029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.368660 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.370328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.377351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 445/907] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:38.389300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.404818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.485863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144749d1-8c37-4fd1-b129-bd3b022d0445 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.503732 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e765b-f618-4401-8b44-9cd18a6b6dcd None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.546511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c684-94cd-4159-964b-38515359f12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.547477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c684-94cd-4159-964b-38515359f12f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.547821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c684-94cd-4159-964b-38515359f12f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.548105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320c684-94cd-4159-964b-38515359f12f None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.550898 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.551822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.552305 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.552922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.554243 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] RBAC: Adding query filter params (interface=public, service_id=11938221f39e4557ab3717c3b72ac53f, region_id=RegionOne) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.554791 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.556569 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c684-94cd-4159-964b-38515359f12f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.556900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.558039 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320c684-94cd-4159-964b-38515359f12f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.560460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.572941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 446/908] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/endpoints?service_id=11938221f39e4557ab3717c3b72ac53f&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:38.574425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-5320c684-94cd-4159-964b-38515359f12f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:38.576128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:38.576936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:38.577704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:38.580486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 463/909] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:38.585591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c684-94cd-4159-964b-38515359f12f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.690149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.715880 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320c684-94cd-4159-964b-38515359f12f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.726056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4cc8cb-181b-4d97-a088-1b210b5f3a83 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.764631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.765851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.766667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.767454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.768629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.769459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.771463 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b2bea-1e55-4a90-a99f-5aa5963cbece None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.775188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.775871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.776160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.776463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.777162 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.778528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.791690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 447/910] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/services => generated 1497 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:57:38.791859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 464/911] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:38.800811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.923756 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abc654-cb1b-4e25-b806-fe9c9c7b465a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:38.974924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:38.976029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.976564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.977044 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.978102 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] RBAC: Adding query filter params (role.id=5d9c4c3566b14a808b2d396e60dab608, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=b37008507f684731b4687e5498da4ef5) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:38.978629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:38.980326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9e91e-343b-40f4-9bb1-7879812db907 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:38.987513 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 448/912] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/role_assignments?user.id=b37008507f684731b4687e5498da4ef5&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=5d9c4c3566b14a808b2d396e60dab608 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:39.087615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448bdbe-3a68-4419-ac83-f137bc643bc5 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.087936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448bdbe-3a68-4419-ac83-f137bc643bc5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.088186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448bdbe-3a68-4419-ac83-f137bc643bc5 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.090012 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 465/913] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:39.096011 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.096322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.096634 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.102308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:39.116113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:39.117574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:39.199778 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 449/914] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:39.223090 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448bdbe-3a68-4419-ac83-f137bc643bc5 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:39.291699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:39.341901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448bdbe-3a68-4419-ac83-f137bc643bc5 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:39.382964 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:39.384131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.384651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.385128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.385985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:39.387658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:39.401309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:39.402284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:57:39.403294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:39.406318 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 466/915] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:39.414189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:39.529805 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f66f4-893b-4cad-93b7-ba11ff304966 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:39.563902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:39.570574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:39.571482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.571946 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.572423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.573326 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:39.573918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:39.575499 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:39.581752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 450/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:39.587874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:39.705665 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085b9dd-7596-4715-9dca-047170d84e1a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:39.731492 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:39.746682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:39.747671 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.748135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.748625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.749546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:39.750030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:39.751603 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68c3c39-f83c-45f2-b774-0faec610891c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:39.758686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 467/917] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 19:57:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:39.769535 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:39.889974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa6f89-4bf7-4cb7-9566-af55fcb0cdf0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:39.929454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:39.930415 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.930878 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.931331 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.932321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:39.933913 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985fad0-a0e8-4ece-a82b-83bc23198b95 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:39.963524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 451/918] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:40.164686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362bd6-20d3-439a-a21b-7a194058a459 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:40.164984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362bd6-20d3-439a-a21b-7a194058a459 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:40.165260 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd362bd6-20d3-439a-a21b-7a194058a459 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:40.166655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 468/919] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:40.172684 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:40.172982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:40.173515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:40.192818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:40.195102 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:40.256587 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:40.287106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 452/920] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:40.310551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd362bd6-20d3-439a-a21b-7a194058a459 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:40.427192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd362bd6-20d3-439a-a21b-7a194058a459 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:40.463499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:40.464601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:40.465026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:40.465441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:40.466278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:40.467915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:40.481815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:40.482854 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:40.483726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:40.486162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 469/921] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:40.492939 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:40.621875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec124b-d6f4-4b49-840d-99e8899e491f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:40.660772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:40.661447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:40.661789 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:40.662069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:40.662780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:40.663089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:40.664481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:40.670219 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 453/922] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:40.676077 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:40.733422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:40.789904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa9d9d-5142-42a8-ba8e-33e17a7f8f02 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:40.826577 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:40.827615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:40.828127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:40.828642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:40.829576 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:40.830133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:40.831845 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:40.839298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 470/923] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:40.844821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:40.899274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:40.963184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4de24-eea1-4052-badd-fe3633bee67a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.001130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.001712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.001998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.002271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.003062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] RBAC: Adding query filter params (interface=public, service_id=4533fff358bb43f1931fc0b378c8211e, region_id=RegionOne) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:41.003373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:41.004769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7345dcb-ca84-4311-8b70-3a155b690311 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:41.011601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 454/924] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:40 2022] GET /identity/v3/endpoints?service_id=4533fff358bb43f1931fc0b378c8211e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:41.016774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.125726 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57f7e5-2f22-4866-b1a0-90f43a7ec112 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.148437 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:41.170936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.171615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.171905 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.172188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.172927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:41.174577 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21d9b6-701b-4d6b-972b-d230372a1c8b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:41.182129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 471/925] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:57:41.412458 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b115-b5ec-493d-8015-d4ec53efaf88 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.412740 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b115-b5ec-493d-8015-d4ec53efaf88 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.413019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b115-b5ec-493d-8015-d4ec53efaf88 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.414673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 455/926] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:41.420559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.420951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.421344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.442187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:41.444004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:41.477447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d621a3-16e5-4171-a81f-e70f54e1d262 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.477798 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d621a3-16e5-4171-a81f-e70f54e1d262 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.478090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d621a3-16e5-4171-a81f-e70f54e1d262 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.479753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 456/927] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:41.485078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.485374 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.485658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.513112 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:41.514352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:41.545373 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 472/928] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:41.574921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.601687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 457/929] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:41.632957 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.671933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:41.699125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42563207-a18a-4cea-9e21-41ecf84ec474 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.710501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:41.739069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.740915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.741407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.741883 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.742741 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:41.744351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:41.746823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12ad59-b0b7-414e-958e-2e08d9235f9d None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:41.757960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:41.758851 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:57:41.759621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:41.767608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 473/930] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:41.769341 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.785141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.785970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.786251 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.786530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.799116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:41.807603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:41.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:41.809727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:41.813964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 458/931] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:41.820231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.886613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bfd77-82d6-4d56-9310-673e1af0e821 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.935385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.935385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.935385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.935385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.935789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:41.935789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:41.935940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:41.938264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:41.943728 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2de465-d356-4a62-963d-11e296fec8ff None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:41.949124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 474/932] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:41.958054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.983872 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:41.984944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.985401 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.985844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.986869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:41.987376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:41.989047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.003400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 459/933] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:42.004679 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.007432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.109585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ae0ed-1b7d-41ed-8f8f-e4968e44abc1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.131403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed80cf-0600-45de-9f59-4c197ac4907a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.163344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.164025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.164313 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.165049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.165936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:42.166352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.167763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.170437 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.171491 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.171997 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.172526 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.176478 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 475/934] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:42.180799 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.190111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.191867 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:42.211038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:42.211891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:42.213144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:42.213144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:42.213144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:42.213144 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:42.219874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 460/935] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:42.221333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.239495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.326112 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.329895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd489db-b085-4802-b70a-ff72dbd97659 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.353349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f0bf0-328a-4b64-b033-9e0d32ba4f18 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.377438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.378678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.378971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.379253 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.380005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.382647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fc711-8803-4267-96c5-427d85d9c149 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.400046 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.401034 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.401440 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.402763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.402949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:42.403273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.404810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.420459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 476/936] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:42 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:57:42.431657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 461/937] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:42.440721 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfbe-d588-47fc-92ea-2cbd00edc651 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.440966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfbe-d588-47fc-92ea-2cbd00edc651 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.441201 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6700bfbe-d588-47fc-92ea-2cbd00edc651 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.442623 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 477/938] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:42.447141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.449511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.449844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.450129 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.469830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:42.471969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:42.546954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.592648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448de31-b66d-4cb6-9094-79db7d3ad1ba None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.592832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 478/939] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:42.614448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.654721 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.656537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.657160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.657456 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.667701 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.669923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:42.690696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:42.691700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:42.692591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:42.694167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 462/940] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:42.702376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.732133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.754352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfe693-484d-4b65-ad55-6c3443b7a1ee None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.795096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.796106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.796619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.797092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.798111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.799951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.801759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: WARNING keystone.server.flask.common [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:57:42.822120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.sql.core [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:57:42.822120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:57:42.822120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [parameters: {'id': 'd454975bfcaf4de5a0e34dbb1ec282bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:57:42.822120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150737) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:57:42.832879 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73f13-73f5-4b8d-9edf-95f1b5a5fe29 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d454975bfcaf4de5a0e34dbb1ec282bc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d454975bfcaf4de5a0e34dbb1ec282bc. May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application conn.query(q) May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:57:42.834494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result.read() May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 19:57:42.835192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application next(self.gen) May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application yield resource May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application next(self.gen) May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise self.value May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application session.commit() May 19 19:57:42.836033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.session.flush() May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application compat.raise_( May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:57:42.836890 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:57:42.837685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application conn.query(q) May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application result.read() May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:57:42.838413 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [parameters: {'id': 'd454975bfcaf4de5a0e34dbb1ec282bc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.839187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:42.839895 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d454975bfcaf4de5a0e34dbb1ec282bc. May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 479/941] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 19:57:42 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:42.848392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:42.887375 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.896709 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.897671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.898135 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.898419 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.899333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:42.899657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.901554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.934449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 463/942] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:42.935814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.004315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7517eeed-464a-4fb7-bb38-a74b74a48c30 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.054037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379aa1-e6d3-45a8-ad1b-91227c3068b5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.057707 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.058766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.059417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.059884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.071179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.074050 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.077110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:43.097650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:43.098515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:43.100036 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:43.106718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 480/943] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:43.107522 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.108339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.109311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.109923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.110209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.131901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.133429 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:43.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:43.154070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:43.154854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:43.154854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:43.154854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:43.154854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:43.159344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 464/944] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:43.160706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.209092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:43.247706 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482c3b-1d13-4326-b455-a9146ef4051f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.284452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94878c-737b-4510-98c4-c82ca5b21da6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.289848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.290789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.291266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.291736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.292818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:43.293338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.295084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.328569 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 481/945] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:43 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:43.331121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.331833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.332124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.332439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.333253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:43.334127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.335563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.354438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 465/946] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:43.356297 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.430781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:43.482107 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1fbfa-df59-42d4-b52d-0202da1d3171 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.528592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.529700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.530121 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.530527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.540466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.542453 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-243b08fe-03bc-4d43-9eb4-ed54a329e13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:43.564801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:43.566430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:43.568684 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:43.570949 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 482/947] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:43.577109 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.687325 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b70c-8a07-48a2-baef-47b1da504c55 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.687645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b70c-8a07-48a2-baef-47b1da504c55 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.687894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325b70c-8a07-48a2-baef-47b1da504c55 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.689593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 483/948] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:43.691905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29833a-54cb-4b41-b816-5d285c72bc17 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.695600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:43.696196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.696749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.696875 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.716587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:43.718233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:43.732434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.733129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.733420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.733701 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.734522 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:43.734839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.736330 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.754775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 466/949] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:43.761786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.832578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 484/950] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:43.856319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:43.889943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f0af0-94cd-45bb-8d6a-d9dff4176db8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:43.912514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:43.934947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:43.935924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:43.936206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:43.936594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/b37008507f684731b4687e5498da4ef5/roles/5d9c4c3566b14a808b2d396e60dab608` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:43.939087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=b37008507f684731b4687e5498da4ef5, role_id=5d9c4c3566b14a808b2d396e60dab608)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:43.940550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:43.968429 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 467/951] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:57:43 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/b37008507f684731b4687e5498da4ef5/roles/5d9c4c3566b14a808b2d396e60dab608 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:57:43.991364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116dccf-a831-4742-b1d8-500de14d560a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:44.034559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:44.035391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:44.035678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:44.035958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:44.036704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:44.038805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:44.053174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:44.054156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:44.054910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:44.055602 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 485/952] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:44.062814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:44.089296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:44.190100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12438e-a370-48a3-8371-b0ffa2ff3f41 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:44.230267 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:44.230949 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:44.231235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:44.231514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:44.232226 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:44.232566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:44.233970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:44.239954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 468/953] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:44.245954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:44.358501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3f1c9-6273-4c5d-8333-ecf1ae2991ed None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:44.374203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:44.399517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:44.400611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:44.401126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:44.401598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:44.402510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:44.403014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:44.404612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bd1736-5a1d-403c-8708-e1f7250a97e1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:44.410651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 486/954] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:44.417195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:44.513510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:44.533043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008e0a8-3784-40a9-bcf7-5d576691e7eb None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:44.570241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:44.570916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:44.571205 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:44.571488 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:44.572254 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:44.573793 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268b222-3411-42b9-abea-ce355bf3d959 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:44.598073 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 469/955] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:45.041614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90faec-dac2-4696-b285-cdec682aba90 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.042714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90faec-dac2-4696-b285-cdec682aba90 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.043202 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90faec-dac2-4696-b285-cdec682aba90 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.045255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 487/956] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:45.052040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.052790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.053291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.071612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:45.073558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:45.154211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 470/957] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:45.178109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90faec-dac2-4696-b285-cdec682aba90 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:45.312015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90faec-dac2-4696-b285-cdec682aba90 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:45.350508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:45.351480 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.351762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.352035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.352825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:45.354567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:45.368449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:45.369424 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:45.370399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 488/958] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:45.377803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:45.456823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:45.489003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f30f64-0ee3-4e82-9f12-7e52c8ef2d4e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:45.532088 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:45.533110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.533600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.534256 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.535166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:45.535736 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:45.537427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70129f66-33c4-4627-9595-ced0c3587019 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:45.547715 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 471/959] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:45.549567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:45.601892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:45.684678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07058b80-5183-4330-bb6a-1972b1d5ce79 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:45.722723 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:45.723789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.725068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.725628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.726589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:45.727093 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:45.728709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0de24-a20f-44cd-a20e-1f30d6a708ce None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:45.740585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 489/960] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:45 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:45.810678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f3285-7887-4c3a-a67c-6fc04aed66c9 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.811331 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f3285-7887-4c3a-a67c-6fc04aed66c9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.811819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f3285-7887-4c3a-a67c-6fc04aed66c9 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.813792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 472/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:45.823455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.824147 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.824669 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.843894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:45.845508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:45.934814 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 490/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:45.956812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f3285-7887-4c3a-a67c-6fc04aed66c9 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:46.077017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f3285-7887-4c3a-a67c-6fc04aed66c9 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:46.112232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:46.119544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:46.120525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.121006 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.121475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.122401 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:46.124018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:46.125625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: WARNING keystone.server.flask.common [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:57:46.140986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.sql.core [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:57:46.140986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:57:46.140986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [parameters: {'user_id': '4a481e976d9b4bf3853c33b3a27184d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:57:46.140986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=150738) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application conn.query(q) May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application result.read() May 19 19:57:46.159351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:57:46.160201 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application next(self.gen) May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application yield resource May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application next(self.gen) May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise self.value May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application session.commit() May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:57:46.161172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self.session.flush() May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application compat.raise_( May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:57:46.162247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:57:46.163062 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application conn.query(q) May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application result.read() May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:57:46.165433 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [parameters: {'user_id': '4a481e976d9b4bf3853c33b3a27184d2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:46.167306 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 19:57:46.169153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:57:46.171168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:46.173766 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 473/963] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 19:57:45 2022] POST /identity/v3/users => generated 160 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:46.180708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:46.223525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:46.301237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42955c4-b42b-4de8-aeae-4152b5c7bc1f None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:46.344370 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:46.345226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.345518 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.345793 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.354579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:46.356064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c4baee00-646b-4fc6-bbd5-a71ca6fb9456 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:46.372030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:46.372955 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:46.374019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 491/964] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:57:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:46.382395 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:46.513133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5553cb-2900-4335-8ebe-4efd960775e3 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.513355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5553cb-2900-4335-8ebe-4efd960775e3 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.513674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5553cb-2900-4335-8ebe-4efd960775e3 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.515745 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 492/965] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:46.520486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.520789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.521144 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.539440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:46.540658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:46.574769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf080fe-756f-49b8-920e-3b1fb2c5a486 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:46.614332 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:46.615106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.615399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.615677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.616537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:46.616870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 493/966] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:46.617017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:46.618507 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:46.637712 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:46.662290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 474/967] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 19:57:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:46.665766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:46.757254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955ee50-4ac4-4b6d-a052-40c9fbdf2586 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:46.797886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:46.798870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.799151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.799428 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.812049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:46.821423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:46.823583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:46.825451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 494/968] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:57:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:46.829130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:46.886659 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:46.944237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e156ad-6e2a-422f-88f5-1095ecd74a22 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:46.984897 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:46.985580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:46.985871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:46.986149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:46.986910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:46.987221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:46.988627 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:46.998015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 475/969] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:57:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:47.003779 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.103670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:47.121172 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3112d833-9e65-4b0b-bb3e-703597c474d3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.159164 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.159991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.160277 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.160588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.173769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.175258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:47.196153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:47.197497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:47.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:47.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:47.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:47.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:47.198338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 495/970] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:47.204656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.317280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f9865-0fb6-45f0-95ec-69788fc41a17 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.331090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:47.356322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.356944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.357255 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.357518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.358252 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:47.358562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.359950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71899f-f3fe-4516-82f3-de681339c490 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.376963 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 476/971] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:47.378192 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.472810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808b1f-baff-4dd8-8b4f-8fb7580d4c69 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.473323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808b1f-baff-4dd8-8b4f-8fb7580d4c69 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.473824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81808b1f-baff-4dd8-8b4f-8fb7580d4c69 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.476628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 477/972] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:47.481759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.482058 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.482342 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.495151 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43212de5-b27a-4c9d-9c96-6b833fe27bb4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.504032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:47.506621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:47.545120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.546245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.546694 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.547149 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.552141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:47.559271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.561041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:47.576811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:47.577708 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:47.578548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 496/973] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:47.584197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.588434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 478/974] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:47.610026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.711594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb07f18-5431-499b-bd9a-6c0ba5cfede0 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.725945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872795d3-ec9a-44aa-8c00-5a8f3b836268 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.727662 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:47.759658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.760390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.760719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.761005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.761863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:47.762182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.763653 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.772615 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.773436 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.773717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.773989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.774645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.776007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.783211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 497/975] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:47.789085 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:47.791286 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:47.792175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:47.792965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 479/976] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:47.798550 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:47.863295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:47.910325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a9cbf-3729-4584-8fe4-bd66bb956517 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.915891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a69362-5973-46ba-9f3e-b9fec008d112 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:47.948797 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.949354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.949717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.949925 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.950630 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:47.950943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:47.952315 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:47.954715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:47.955461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:47.955889 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:47.956182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:47.958367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 480/977] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:47.964985 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.001668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.046276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.047756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.101881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e9ba2-9eee-4b31-99b4-14fac9537436 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.109180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.149827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.150650 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.150831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.151086 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.151835 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:48.152205 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.153644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.159469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 481/978] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:48.167418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.190588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 498/979] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:57:48.197187 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.287753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.295166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80963313-7a94-4fbd-bc60-73f32f89fc5e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.315044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d785a43-5e82-453f-820a-c19a02af6080 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.342536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.343203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.343489 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.343774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.344594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] RBAC: Adding query filter params (interface=admin, service_id=4533fff358bb43f1931fc0b378c8211e, region_id=RegionOne) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:48.344919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.346296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2a236-a420-4091-84ea-c3cbd3288ca1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.352175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 482/980] 10.222.0.13 () {56 vars in 1249 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/endpoints?service_id=4533fff358bb43f1931fc0b378c8211e&interface=admin®ion_id=RegionOne => generated 193 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:48.353459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbb76-69f0-427a-882e-a93a5d63a779 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.353755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbb76-69f0-427a-882e-a93a5d63a779 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.354031 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbb76-69f0-427a-882e-a93a5d63a779 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.355088 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.355308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 483/981] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:48.356140 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.356453 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.356741 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.357842 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cbb76-69f0-427a-882e-a93a5d63a779 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.368352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.369847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f92ecded-3884-4b65-8638-7b751e863a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:48.388891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:48.389776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 499/982] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.395606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.403826 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.410581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:48.412489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:48.467907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cbb76-69f0-427a-882e-a93a5d63a779 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.481879 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 500/983] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:48.482763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.514841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.515518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.515811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.516091 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.516818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.518212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.525567 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 484/984] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:57:48.526373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.557561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.617664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d200b-2a9f-4931-a1e5-3fd62f1c83a0 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.658994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff521e87-3324-4c76-bf01-2a587689bac7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.667161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.668605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.668899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.669183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.669952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:48.670725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.672594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.676518 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:48.692459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 501/985] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:48.694521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.710281 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.711132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.711423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.711762 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.712668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.713969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.731001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 485/986] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:48 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:48.816316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e7cb2-d505-4897-b711-f309ca6227f3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.865661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.866600 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.866893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.867177 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.875099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:48.876626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:57:48.902610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:48.903519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:48.904321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 502/987] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:48.910890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:49.014298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:49.038590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599f888-20f7-4531-8d6c-af0904425c9d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:49.078143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:49.078873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:49.079167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:49.079447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:49.080239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:49.080585 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:49.083632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6e13-a7f9-4576-b2bf-0b1acdb88319 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:49.095726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 486/988] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:49.101900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:49.161045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:49.213611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d18226-2d6b-43e9-bcf1-eddf06b8a5d7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:49.251987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:49.252689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:49.252973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:49.253249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:49.254191 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] RBAC: Adding query filter params (role.id=5d9c4c3566b14a808b2d396e60dab608, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=b37008507f684731b4687e5498da4ef5) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:49.254503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:49.256203 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439371de-a36b-4406-aaea-1c3b45193d8f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:49.263656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 503/989] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:57:49 2022] GET /identity/v3/role_assignments?user.id=b37008507f684731b4687e5498da4ef5&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=5d9c4c3566b14a808b2d396e60dab608 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:50.827591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4253d9-3ef5-4077-8c67-c9dbfa2e810b None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:50.828092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4253d9-3ef5-4077-8c67-c9dbfa2e810b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:50.828092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4253d9-3ef5-4077-8c67-c9dbfa2e810b None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:50.829783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 487/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:50.836324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbead5-3115-40a9-a888-28b116dc91ab None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:50.836803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbead5-3115-40a9-a888-28b116dc91ab None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:50.836989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbead5-3115-40a9-a888-28b116dc91ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:50.855259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-3acbead5-3115-40a9-a888-28b116dc91ab None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:50.856529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-3acbead5-3115-40a9-a888-28b116dc91ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:50.944548 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 504/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:50.968215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4253d9-3ef5-4077-8c67-c9dbfa2e810b None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.052336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676c7e-91cc-4e97-885d-dcf5771f8aa7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.052737 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676c7e-91cc-4e97-885d-dcf5771f8aa7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.053029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad676c7e-91cc-4e97-885d-dcf5771f8aa7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.054616 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 505/992] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:51.059716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.060030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.060426 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.077943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:51.079398 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:51.094688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4253d9-3ef5-4077-8c67-c9dbfa2e810b None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.129116 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.137293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.138162 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.138450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.138731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.139426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.140903 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:51.154586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:51.155590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:51.156333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 488/993] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 19:57:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:51.162342 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.172892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 506/994] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:51.173064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.194773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.279464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6edc83d-051a-4e34-807f-98452183da9a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.304688 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b855a7-fa44-45a1-ba30-e8448a02a30e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.322408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.323080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.323376 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.323660 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.324381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:51.324753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.326113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.332710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 489/995] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:51.338069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.345392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.346669 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.347213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.347747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.348473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.350680 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:51.364324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:51.365801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:51.366547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:51.367195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 507/996] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:51.372969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.419223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.430295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.465212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3030313-a76e-4dc9-b180-45728de61cea None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.502709 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fc66d-89d5-4e10-b2f5-c4ac72add440 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.503966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.504748 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.505047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.505410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.506933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:51.507258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.508710 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.519244 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 490/997] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:51.520258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.549713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.550400 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.550687 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.550966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.551776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:51.552096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.554970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.565299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 508/998] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:51.566175 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.644515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc80dfe-b7f0-47fd-9540-0c5321e7b045 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.681487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.682260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.682555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.682836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.683580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.683872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63919c22-db79-4ce6-b512-1fdf09e72e23 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.685019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64312c30-9fa3-4541-a3a6-b846a7f0cc60 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.686867 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.689216 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.704003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 491/999] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:57:51.705372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b466-d05e-4736-93fa-6f19626cd71b None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.705662 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b466-d05e-4736-93fa-6f19626cd71b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.705943 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532b466-d05e-4736-93fa-6f19626cd71b None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.707462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 492/1000] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:51.715479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.715771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.716053 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.734410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:51.734770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.735397 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.735673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:51.735769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.735941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.736707 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:51.737020 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.738440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.745153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 509/1001] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:51.751781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.825092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 493/1002] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:51.844077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.882935 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275f76d-19fd-4c6b-9736-033a90bf62e7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.934056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:51.934658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:51.934948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:51.935228 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:51.936033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] RBAC: Adding query filter params (interface=public, service_id=0d9e4c564bf24e70aaa181302de843f7, region_id=RegionOne) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:51.936611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:51.937811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:51.944535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 510/1003] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/endpoints?service_id=0d9e4c564bf24e70aaa181302de843f7&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:51.950285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:51.971618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e38946-4a6c-4672-821b-e7c29f07ff8f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:51.985545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:51.999619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:52.015122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:52.016009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.016358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.016731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] PATH_INFO: `/v3/services/image` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.017459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:52.018947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:52.032424 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:52.033405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:52.034147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 494/1004] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:52.039944 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:52.075966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f2daa-7b87-4ae3-9b91-99c620a3e7ab None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:52.115351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:52.116035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.116307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.116617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.117303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:52.118794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:52.128783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 511/1005] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:51 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:57:52.150982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa05b3-090b-4e25-b6f6-5cfe543d81de None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:52.191845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:52.192482 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.192798 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.193082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.193833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] RBAC: Adding query filter params (name=image) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:52.194141 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:52.195561 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca4b46-74ab-48a4-9fb3-fbfbd1c14bbb None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:52.201478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 495/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:52.206912 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:52.316095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:52.327311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561de179-2f75-4140-913f-f37ac87d902d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:52.391433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:52.392138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.392443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.392720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.396224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] RBAC: Adding query filter params (type=image) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:52.396564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:52.397966 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef40c-ce55-4566-88cb-01a7076bd2b7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:52.408475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 512/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:54.018438 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e712-aad0-4c2f-b909-0255d84194f6 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.019039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e712-aad0-4c2f-b909-0255d84194f6 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.019039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35e712-aad0-4c2f-b909-0255d84194f6 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.020562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 496/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:54.026747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.027198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.027399 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.045874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:54.047096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:54.123695 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 513/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:54.146916 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35e712-aad0-4c2f-b909-0255d84194f6 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:54.206199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:54.259865 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35e712-aad0-4c2f-b909-0255d84194f6 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:54.308947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:54.309705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.309920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.310199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.311406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:54.312903 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799722e-e6db-49a0-9107-56da935db012 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:54.320242 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 497/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:54.321471 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:54.426455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:54.446557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e51fad-81cf-4de7-8ee2-117fa1691c44 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:54.469574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982286a-3786-4be5-925d-b010ac0354c2 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.469884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982286a-3786-4be5-925d-b010ac0354c2 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.470146 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982286a-3786-4be5-925d-b010ac0354c2 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.471779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 498/1011] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:54.478510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00785-55e0-4280-9535-ac6ca8321ffe None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.478810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00785-55e0-4280-9535-ac6ca8321ffe None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.479092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c00785-55e0-4280-9535-ac6ca8321ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.494733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:54.495436 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.495725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.496010 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.496802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:54.497837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-b3c00785-55e0-4280-9535-ac6ca8321ffe None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:54.498322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:54.498959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-b3c00785-55e0-4280-9535-ac6ca8321ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:54.531940 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 514/1012] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:54 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:57:54.608528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 499/1013] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:54.640654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:54.697599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f23a2-6917-4f10-b53e-5316f3405590 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.697947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f23a2-6917-4f10-b53e-5316f3405590 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.698171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f23a2-6917-4f10-b53e-5316f3405590 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.699611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 500/1014] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:54.705537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.705832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.706104 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.724543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:54.725964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:54.767351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e222b-9c94-4751-927e-4edcd0e9cfce None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:54.783851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:54.810392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 501/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:54.819441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:54.820253 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.820595 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.820884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.821551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:54.822945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:54.833924 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:54.836179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:54.837275 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:57:54.838014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:54.839274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 515/1016] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:54.843753 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:54.846842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:54.949392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a3fd4-8924-4f07-a2c5-4f11fb8df8a8 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:54.969604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20602591-1174-40d5-be0a-ccc793ae4be1 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:54.992792 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:54.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:54.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:54.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:54.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:54.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:55.014970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:55.015656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.015927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.016206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.016951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] RBAC: Adding query filter params (name=compute) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:55.017441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 502/1017] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:57:54 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:57:55.024715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:55.026283 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:55.038349 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 516/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:55.039617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:55.089364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:55.178179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccc8b5-bdcc-43be-ad79-4410fda704be None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:55.219935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:55.220569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.220861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.221147 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.221863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] RBAC: Adding query filter params (type=compute) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:55.222176 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:55.223546 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c42b74-329a-4160-83fa-10542dd4a149 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:55.229594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 503/1019] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 19:57:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:55.235320 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:55.311877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:55.351232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee5bcc-ef76-4794-b82e-7cb154864e01 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:55.388220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:55.388922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.389213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.389494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.390301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:55.391641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231f6e0-49f7-484c-9166-ecaaa3c9cc78 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:55.424431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 517/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:57:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:56.888896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80700-f512-47e5-ab55-ca530ccf36c1 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:56.889403 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80700-f512-47e5-ab55-ca530ccf36c1 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:56.889478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db80700-f512-47e5-ab55-ca530ccf36c1 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:56.890998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 504/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:56.896731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:56.897246 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:56.897344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:56.914593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:56.915795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:57.000045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 518/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:57:57.026001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db80700-f512-47e5-ab55-ca530ccf36c1 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:57.140577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:57.146874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db80700-f512-47e5-ab55-ca530ccf36c1 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:57.194120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:57.194945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.195234 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.195513 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.196768 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:57.198240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed11467-2b40-482f-9906-1b6a62b1b286 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:57.200080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 505/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:57.205626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:57.267642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:57.332819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecb211-cd73-4770-83a2-ccce320bfc8f None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:57.372972 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:57.373656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.373946 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.374226 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.374960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:57.376442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddc44b-f6eb-4e25-8119-c1d421e206d4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:57.413330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 519/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:57:57 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:57:57.429784 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be769a-51fa-4b48-bda5-1ea54fb76d0f None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.430432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be769a-51fa-4b48-bda5-1ea54fb76d0f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.430935 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be769a-51fa-4b48-bda5-1ea54fb76d0f None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.433955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 506/1025] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:57.440319 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.440662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.440884 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.480044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:57.482130 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:57.594255 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 520/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:57.618614 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be769a-51fa-4b48-bda5-1ea54fb76d0f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:57.766035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be769a-51fa-4b48-bda5-1ea54fb76d0f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:57.813562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:57.814774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.815198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.815608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] PATH_INFO: `/v3/services/image` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.816569 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:57.818532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-61e52566-e285-40b6-8af8-7b4f2842aeb1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:57.831824 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:57.832818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:57.834402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 507/1027] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:57:57 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:57.841219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:57.847471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50e3-d2a8-4267-9f5e-a7b741f3b7da None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.847764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50e3-d2a8-4267-9f5e-a7b741f3b7da None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.848048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce50e3-d2a8-4267-9f5e-a7b741f3b7da None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.849770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 508/1028] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:57:57.855462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:57.855919 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:57.856366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:57.876502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:57.878024 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:57.879440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:57.909891 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:57.964563 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 509/1029] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:57:57.966999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5b93a-f7c4-496f-a803-c5f7cd1fb1f2 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:57.988163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.014886 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.016517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.017079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.017503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.018531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] RBAC: Adding query filter params (name=image) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:58.019273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.022516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.029290 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 521/1030] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:58.035414 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.111278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c48904-8eff-4a98-b3d3-f33a48ad44b5 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.149502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.153094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cede9221-3f3c-48b1-9f78-2fa7b9a54c9a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.153631 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.154459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.154962 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.155205 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.169677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:57:58.181022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:58.181983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:58.182755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 510/1031] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:57:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:58.188772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.194064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.194064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.194288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.194513 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.195234 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] RBAC: Adding query filter params (type=image) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:58.195542 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.196983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.202673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 522/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:58.208681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.221061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.313618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a59e7-831d-4670-9f10-a515452437b1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.323803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e59a-d2e6-422d-ac3b-0ef9c541d263 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.354851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.355526 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.355818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.356105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.356894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:58.357209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.358608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.361499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.362161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.362446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.362728 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.363526 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] RBAC: Adding query filter params (interface=public, service_id=db64fdeb11354f0898a949dd1c9cfb04, region_id=RegionOne) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:58.363842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.365788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.371660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 523/1033] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/endpoints?service_id=db64fdeb11354f0898a949dd1c9cfb04&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:57:58.376027 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 511/1034] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:58.377417 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.377540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.413655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.471759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.488920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583150d-5279-4146-9f3a-ddc222429d76 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.498998 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474d9bb-0adf-4e88-8a91-e9abb4e8ca86 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.533259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.533940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.534406 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.534586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.535265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.537180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.543855 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.553009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.553237 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 512/1035] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:57:58.553329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.553329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.563891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.565546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:58.587384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:58.588338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:58.589391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:58.589391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:58.589391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:58.589391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:58.590142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 524/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:58.595772 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.748132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd1159-3f07-46af-93b2-9e40c2b4631e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.788060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.788905 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.789198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.789552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.790485 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:58.791077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.793344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.812823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 513/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:58.819839 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.917474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.943693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f6614-904e-4462-a547-f116b8978c2b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.984196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4324-e834-4017-8082-01f26025543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.985072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4324-e834-4017-8082-01f26025543c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.985358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4324-e834-4017-8082-01f26025543c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.985633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca4324-e834-4017-8082-01f26025543c None admin] PATH_INFO: `/v3/users/nova` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.993971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca4324-e834-4017-8082-01f26025543c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.995456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca4324-e834-4017-8082-01f26025543c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-34ca4324-e834-4017-8082-01f26025543c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:57:59.011057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:59.012125 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:57:59.013973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:57:59.016489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 525/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:57:59.018413 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:59.031288 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:59.142714 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a08321-c989-48e9-83e9-97cfb69ec9dc None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:59.182555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:59.183323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:59.183610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:59.183976 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:59.184809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:59.185128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:59.186584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:59.211223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 514/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:57:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:59.212711 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4324-e834-4017-8082-01f26025543c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:59.332181 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca4324-e834-4017-8082-01f26025543c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:59.373465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:59.375069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:59.375069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:59.375069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:59.384885 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:59.469547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:59.470936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:59.594386 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 526/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:57:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:57:59.600183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:59.681940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:59.722110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c1747-a4ed-48b0-8822-4ef2eaa528df None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:59.758873 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:59.759679 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:59.759965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:59.760241 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:59.770869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:59.772339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:57:59.791625 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:57:59.792932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:57:59.798970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:57:59.798970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:57:59.798970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:57:59.798970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:57:59.798970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 515/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:57:59.799609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:59.844346 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:59.912869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e936f5ed-3d91-4c32-a204-ba1d23e93952 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:59.949180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:59.949891 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:59.950179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:59.950455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:59.951198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:57:59.951506 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:59.953086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9692377-c2c1-411f-8c1e-198e1da9ad88 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:59.971549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 527/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:57:59.973037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:59.992750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b718-4c33-4e89-a980-9c07340485bb None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:59.993376 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b718-4c33-4e89-a980-9c07340485bb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:59.993580 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b718-4c33-4e89-a980-9c07340485bb None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:59.995512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 528/1043] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:57:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:58:00.000451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.000791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.001046 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.023572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:00.024926 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:00.102215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5414f9a-01bb-4d55-b0e9-b8e7137741c3 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.108157 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 529/1044] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:57:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:58:00.124215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:00.132072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.148555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.149637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.149929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.150216 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.163461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:00.165755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.167478 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:58:00.183618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:00.184810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:00.185671 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 516/1045] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:57:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:00.190473 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.256662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8adbb-025c-4bbd-a5ac-7db4cfe929df None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.300045 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749b5e4-598f-4bb4-8a60-ac2db6bee275 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.300266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.301072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.301354 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.301643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.313512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:00.321999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:00.322994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:00.323800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 530/1046] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:00.329714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.346285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.346951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.347238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.347517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.348304 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:00.348643 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.350319 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:00.363057 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 517/1047] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:00.369534 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.413019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:00.431870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:00.446391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e874f3-9502-4015-a8b6-ee10756ba1f9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.489247 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97519a3e-fdb0-4105-8917-ad3d79870023 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.491953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.492679 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.493392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.493392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.494158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:00.494476 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.495922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:00.509241 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 531/1048] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:00.514769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.546419 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.547087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.547379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.547658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.548776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] RBAC: Adding query filter params (role.id=5d9c4c3566b14a808b2d396e60dab608, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:00.549098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.550548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:00.557378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 518/1049] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=5d9c4c3566b14a808b2d396e60dab608 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:00.659516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a3b88-86e9-4483-b373-c3b2341d7e3c None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.709804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.710727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.711040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.711289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] PATH_INFO: `/v3/users/a3767ca8648449ef9971ee7ea9a713ce` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.729037 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] RBAC: Authorizing `identity:get_user(user_id=a3767ca8648449ef9971ee7ea9a713ce)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.730538 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f44b03-c350-4e53-842e-1238269149c1 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:00.732557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 532/1050] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/users/a3767ca8648449ef9971ee7ea9a713ce => generated 285 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:00.746357 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:00.869155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a31c01-2132-448a-9fe2-6be8e2a3ab7c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:00.870764 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:00.915938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:00.916632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:00.916926 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:00.917209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:00.997549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:00.999075 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.001167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a27a-c992-47f2-82a7-3d2f67e62085 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.001638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a27a-c992-47f2-82a7-3d2f67e62085 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.001723 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a27a-c992-47f2-82a7-3d2f67e62085 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.003145 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 533/1051] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:01.009494 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.009783 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.010066 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.028300 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:01.029549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:01.031682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.109501 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 534/1052] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:58:01.133237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.153408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 519/1053] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:01.160393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.164450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.261126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6128d6-5a25-4699-bb96-4b3e01679154 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.277809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd7b81-d681-46c6-9850-52dabab79a2d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.304303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.311950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.312335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.312640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] PATH_INFO: `/v3/services/image` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.313348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.314780 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.317504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.318936 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.319321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.319603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.320884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.322385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.324361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 520/1054] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:01 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:01.330524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:01.330834 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:01.331724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:01.337784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 535/1055] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 19:58:01 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:01.340011 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.342848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.409410 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.445777 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda15761-5d30-49a1-b45f-acbf3bee19d4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.462559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa2a59-55fe-42ff-9c79-91d3f3d39fd7 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.488333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.489019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.489303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.489731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.490593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=6e5bf08e446647c3a3d14381824dd064, user.id=a3767ca8648449ef9971ee7ea9a713ce) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:01.490909 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.492344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.512435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 521/1056] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:01 2022] GET /identity/v3/role_assignments?user.id=a3767ca8648449ef9971ee7ea9a713ce&scope.project.id=6e5bf08e446647c3a3d14381824dd064&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:01.512628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.512628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.512628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.512628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.512911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] RBAC: Adding query filter params (name=image) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:01.512911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.512911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.518907 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 536/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:01.525265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.657105 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2a400-9c4e-4481-84e4-d9dfc4aeafb8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.697295 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.697896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.698184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.698467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.699194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] RBAC: Adding query filter params (type=image) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:01.699517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.700959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c1de-8caf-4c6e-8fbf-d0ba5fbe0c1d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.702470 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.707711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 522/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:01.714034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:01.796213 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:01.834843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af59a27-1e70-4d9f-a15e-caedbc051a6d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:01.870240 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:01.870914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.871199 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.871475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.872232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:01.873648 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb9b22-4ce9-4885-a1b8-432b477e2eaf None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:01.900440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 537/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 19:58:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:02.885126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f372-d3bf-4d8e-9f97-10529fd17134 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:02.885126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f372-d3bf-4d8e-9f97-10529fd17134 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:02.885759 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9f372-d3bf-4d8e-9f97-10529fd17134 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:02.887039 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 523/1060] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:02.892009 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:02.892423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:02.892606 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:02.909971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:02.911179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:02.991362 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 538/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:03.014029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f372-d3bf-4d8e-9f97-10529fd17134 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.126462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9f372-d3bf-4d8e-9f97-10529fd17134 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:03.171705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:03.172559 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.172848 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.173124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.185529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:03.194346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:03.195308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:58:03.196106 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:03.203839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 524/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:03.203990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.257617 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:03.316061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b555d-a010-40f1-a7da-7ed85e251a7f None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:03.355535 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:03.356231 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.356525 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.356803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.357561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:03.357868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:03.359404 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:03.372971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 539/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:03.374586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.443421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:03.489292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c3b4e-c7ea-4e79-a6e0-d6b197d7b1e3 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:03.531186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:03.532003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.532285 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.532597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.544528 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:03.555066 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:03.574121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:03.575035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:03.580689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:58:03.580689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:03.580689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:03.580689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:03.580689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 525/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:03.582667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.700006 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f1fa6-de46-4b8d-b547-83ad407f0994 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:03.712379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:03.742395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:03.743092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.743385 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.743668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.744465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:03.744785 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:03.746266 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65494a-af8f-4ae9-be6d-b7947287629e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:03.774827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 540/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:03.776719 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.887263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fee4c6-0fcc-405e-b907-8ccc1074d474 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:03.898486 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:03.926720 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:03.928153 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.928558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.928850 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.937421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:03.938734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:03.938983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510799d-51a7-4f44-b3ff-8d0be9961db7 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.939422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510799d-51a7-4f44-b3ff-8d0be9961db7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.939632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510799d-51a7-4f44-b3ff-8d0be9961db7 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.941014 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 541/1066] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:58:03.946186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:03.946473 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:03.946764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:58:03.955183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:03.956052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:58:03.956972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:03.960548 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 526/1067] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:03.964687 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:03.969781 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:03.971051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:04.051658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 542/1068] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:58:04.075969 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.086705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a984-90df-425a-adbd-85dcbc6dc59f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.125618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.126460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.126748 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.127023 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.127998 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:04.128451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.129899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.155725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 527/1069] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:58:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:04.156639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.179973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.184244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9508c-be62-489b-ad4b-b0dd1df409d4 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.188361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.221450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.222222 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.222508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.222782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.234846 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a8110878-a49b-446a-a837-9d81bdccda4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:04.243553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:04.244632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:04.245402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 543/1070] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:04.246422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e345-7fba-4989-89fe-7c7d3a407f84 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.246422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e345-7fba-4989-89fe-7c7d3a407f84 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.246654 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0e345-7fba-4989-89fe-7c7d3a407f84 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.247866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 544/1071] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:04.250642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0e345-7fba-4989-89fe-7c7d3a407f84 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.270536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55f103-44d0-41b4-a1ba-d36f27a17532 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.311293 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.312090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.312373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.312689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.323746 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.325235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e1801ac5-0c10-4847-9fed-c3c0ef1b6ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:04.343801 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:04.344904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:04.345743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:58:04.345743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:04.345743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:04.345743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:04.345743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 528/1072] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:04.346468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.346763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.347137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.361759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0e345-7fba-4989-89fe-7c7d3a407f84 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.364307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:04.366040 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:04.398469 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.399146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.399434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.399717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.400506 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:04.400821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.402215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.416303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 545/1073] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:04.416303 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.439857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 529/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:04.441179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.445542 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.465938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.554730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b635f2-8758-451d-b8f5-959192479a02 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.573313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7c7fd-d06e-4853-9573-0e7392460c80 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.596929 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.597614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.597898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.598176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.599554 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:04.599866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.601681 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.614337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 546/1075] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:04.615819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.621549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.622437 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.622722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.622994 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] PATH_INFO: `/v3/users/a3767ca8648449ef9971ee7ea9a713ce` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.628907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] RBAC: Authorizing `identity:get_user(user_id=a3767ca8648449ef9971ee7ea9a713ce)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.630472 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.636872 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 530/1076] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/users/a3767ca8648449ef9971ee7ea9a713ce => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:04.636872 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.725965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.739173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.747029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c95fff-97bf-409f-9e6a-f15eb2867afe None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.767292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd1c2-fed1-4ee0-80a6-56f051e5f099 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.786789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.787470 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.787830 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.788176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.807627 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.808479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.808770 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.809072 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.816925 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.818745 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:58:04.832582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:04.833611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:04.834637 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 531/1077] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:04.835312 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.868557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.870041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.940866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e32d3f-902e-435a-94f9-cc3403de9fe8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:04.954628 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:04.978303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:04.979167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.979448 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.979725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.980932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:04.982370 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:04.984076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 532/1078] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:04.989254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 547/1079] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:04.989442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.990097 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:04.997657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.094315 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c57b5-4d34-430a-a1e4-8999f8440012 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.107505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fac14-7647-433f-936b-786b9ce6de6a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.132629 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.133299 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.133588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.133866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.134658 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:05.134970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.136431 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.144750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.145664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.146232 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.146711 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064/users/a3767ca8648449ef9971ee7ea9a713ce/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.148625 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] RBAC: Authorizing `identity:create_grant(project_id=6e5bf08e446647c3a3d14381824dd064, user_id=a3767ca8648449ef9971ee7ea9a713ce, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.149366 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 533/1080] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:58:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:05.150068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.150275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:05.167541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 548/1081] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:58:04 2022] PUT /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064/users/a3767ca8648449ef9971ee7ea9a713ce/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:58:05.167541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:05.220544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.245466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.285539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bfe17-27d7-4b87-9bbb-be4595ab9751 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.304750 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80b48e-8f9f-40f6-ae94-de14e4b36f45 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.326796 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.327635 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.327922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.328202 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.340597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.342092 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.349758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.350738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.351176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.351462 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/5d9c4c3566b14a808b2d396e60dab608` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.354034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=482a2286f928483b8bce71f07fb4809f, role_id=5d9c4c3566b14a808b2d396e60dab608)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.355517 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b756282e-a95a-4e9b-a851-3feaa84b2b1a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:05.361974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:05.363070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:05.363906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:58:05.363906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:05.363906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:05.363906 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:05.364508 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 534/1082] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:05.371966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:05.379511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 549/1083] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:58:05 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/482a2286f928483b8bce71f07fb4809f/roles/5d9c4c3566b14a808b2d396e60dab608 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:58:05.500283 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee9535-f8f2-4589-80b0-67dba2530074 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.526194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.549185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.549857 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.550136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.550408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.551145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:05.551451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.553004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a78d32-470a-46e5-b594-a3a9c56ca632 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.572452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 535/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:07.477195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785547ca-1bf0-49c4-8a87-7c8af4bea1f2 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.477747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785547ca-1bf0-49c4-8a87-7c8af4bea1f2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.477747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785547ca-1bf0-49c4-8a87-7c8af4bea1f2 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.479323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 550/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:07.484560 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6f79-502d-4ddd-90dc-a9123507560d None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.484951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6f79-502d-4ddd-90dc-a9123507560d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.485137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e6f79-502d-4ddd-90dc-a9123507560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.501990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-599e6f79-502d-4ddd-90dc-a9123507560d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:07.503215 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-599e6f79-502d-4ddd-90dc-a9123507560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:07.580146 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 536/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:07.604898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785547ca-1bf0-49c4-8a87-7c8af4bea1f2 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:07.713329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785547ca-1bf0-49c4-8a87-7c8af4bea1f2 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:07.713920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee1b7-cb69-4d8a-bc7f-99af38c4143b None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.714222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee1b7-cb69-4d8a-bc7f-99af38c4143b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.714506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aee1b7-cb69-4d8a-bc7f-99af38c4143b None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.715208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:07.716131 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 537/1087] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:07.723869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.724259 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.724754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.741773 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:07.742899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:07.752598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:07.753398 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.753685 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.753957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.764865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:07.773570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:07.774727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:07.775540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 551/1088] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:07.780837 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:07.813483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 538/1089] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:58:07.841777 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:07.883626 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:07.901430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43800e-c8a4-4dac-b90a-4978a5aa3ba4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:07.937347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:07.938019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.938297 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.938570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.939316 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:07.939619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:07.941030 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:07.949375 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc42c2-8698-486c-83a0-e773b2a881b8 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:07.949951 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 552/1090] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:07.955930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:07.984766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:07.986163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:07.986990 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:07.987271 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:07.987547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:07.998206 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:08.006459 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:08.007378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:08.008188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 539/1091] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 19:58:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:08.013675 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.069861 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d96bb9-9789-4a51-b438-ed3cf9433a6e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.107911 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.108831 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.109117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.109389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] PATH_INFO: `/v3/users/a3767ca8648449ef9971ee7ea9a713ce` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.110588 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] RBAC: Authorizing `identity:get_user(user_id=a3767ca8648449ef9971ee7ea9a713ce)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.112042 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.113710 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 553/1092] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:07 2022] GET /identity/v3/users/a3767ca8648449ef9971ee7ea9a713ce => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.119570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.122099 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.125899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3463-3b8a-4e60-8c27-e54ce6e7194e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.167391 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.168069 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.168358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.168670 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.169432 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:08.169745 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.171117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.180158 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 540/1093] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.185799 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.220185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.229391 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be72ad-9752-4488-8c8c-df279674a8df None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.267047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.267620 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.267931 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.268209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.293699 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a5ce-65e5-41b6-9cc5-ba2b88afd8b7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.329899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.330705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.330995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.331275 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.339328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.340848 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.342677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.344173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:08.363661 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:08.365272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:08.366101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:58:08.366101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:08.366101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:08.366101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:08.366101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 541/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:08.371396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.387586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.441760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.456597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 554/1095] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:58:08.462832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.479656 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63029c2-f208-4b7c-ba9b-6edc1b65a5a0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.515400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.516038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.516132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.516352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.517183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:08.517440 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.518845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.531316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 542/1096] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.537415 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.589939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353eea8-38d4-4952-893f-996dc92a7a76 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.616992 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.627347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.628219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.628529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.628805 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.630072 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.631518 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.633262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 555/1097] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.639707 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.649434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98751dbc-b47e-4a98-9139-788d5f43e06e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.687227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.688026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.688316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.688617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.696099 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.697580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.705342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:58:08.713090 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:08.714248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:08.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 543/1098] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:58:08.720222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.757902 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7b84d-8501-4def-8a25-6a8bc52dea98 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.798029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.798841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.799123 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.799392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.800220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=6e5bf08e446647c3a3d14381824dd064, user.id=a3767ca8648449ef9971ee7ea9a713ce) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:08.800564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.802140 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.808505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 556/1099] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/role_assignments?user.id=a3767ca8648449ef9971ee7ea9a713ce&scope.project.id=6e5bf08e446647c3a3d14381824dd064&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.824214 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb14b03f-407a-47d9-b19f-0464c35ec5e1 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.858649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.864647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.865493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.865788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.866079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.866982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:08.867301 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.868875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.889837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 544/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:08.896502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:09.010788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7415a-9408-4ca1-a1f4-689112a39f53 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:09.037662 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:09.050049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:09.050633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:09.050914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:09.051191 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:09.130802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:09.132311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:09.238118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 557/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:09.244428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:09.351002 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1990-8635-4520-b5d6-abb2ce6eeb3e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:09.385929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:09.386755 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:09.387035 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:09.387308 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:09.399140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:09.400618 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:09.418929 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:09.419825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:09.420691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:58:09.420691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:09.420691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:58:09.420691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:09.420691 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 545/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:09.426186 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:09.452095 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:09.533765 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0228a6-5e81-4ef0-b3f0-2d37dda5f527 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:09.578247 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:09.578916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:09.579204 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:09.579481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:09.580211 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:09.580555 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:09.581968 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:09.594244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 558/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:09.601266 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:09.650128 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:09.712166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb630f-aa6c-4a1e-a0ed-f0b37e19e814 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:09.748639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:09.749441 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:09.749726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:09.750004 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:09.763233 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:09.764733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:58:09.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:09.784464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:09.785810 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 546/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 19:58:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:09.790733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:09.879689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:09.897651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d4cc3-9a03-40fd-bc57-3b99c3d50a1a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:09.935227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:09.935885 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:09.936167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:09.936468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:09.937259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:09.937562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:09.938991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c34001-58bf-4278-a50c-291b80822f32 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:09.952237 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 559/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 19:58:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:09.957991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:10.063352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dab4d1-e088-4d79-bd5d-72e1f3356d66 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:10.064940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:10.101907 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:10.102580 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:10.102869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:10.103152 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:10.103992 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] RBAC: Adding query filter params (role.id=5d9c4c3566b14a808b2d396e60dab608, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=482a2286f928483b8bce71f07fb4809f) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:10.104312 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:10.105781 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf9a5a-dc37-46a6-93e4-4472ec7cb477 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:10.111921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 547/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:09 2022] GET /identity/v3/role_assignments?user.id=482a2286f928483b8bce71f07fb4809f&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=5d9c4c3566b14a808b2d396e60dab608 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:11.160866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2acd-5802-4147-86b2-5b66f9da8a2d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:11.161406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2acd-5802-4147-86b2-5b66f9da8a2d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:11.161511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2acd-5802-4147-86b2-5b66f9da8a2d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:11.162958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 560/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:11.169623 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:11.170166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:11.170166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:11.188321 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:11.189553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:11.276009 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 548/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:11.301626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf2acd-5802-4147-86b2-5b66f9da8a2d None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:11.410016 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf2acd-5802-4147-86b2-5b66f9da8a2d None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:11.446934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:11.447661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:11.447950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:11.448230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:11.449492 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:11.450942 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feeb420-e93c-49bb-ad1f-ca4fc3bd3246 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:11.452608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 561/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:11.459257 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:11.560726 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:11.567119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83a929-8c71-4e9e-b38e-67ea3cebcfd5 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:11.604167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:11.604869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:11.605159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:11.605430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:11.606140 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:11.607564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b062b15-de7f-489b-964e-a01fd84b9771 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:11.628610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 549/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:58:11 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:13.977618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b5bf-1845-491f-ab0b-cdaea23b59ba None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:13.978180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b5bf-1845-491f-ab0b-cdaea23b59ba None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:13.978180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70b5bf-1845-491f-ab0b-cdaea23b59ba None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:13.979627 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 562/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:13.985616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:13.986120 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:13.986217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:14.002651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:14.003837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:14.073668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 550/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:14.098416 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b5bf-1845-491f-ab0b-cdaea23b59ba None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:14.114097 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:14.206899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70b5bf-1845-491f-ab0b-cdaea23b59ba None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:14.241914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:14.242699 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:14.242989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:14.243261 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] PATH_INFO: `/v3/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:14.244801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] RBAC: Authorizing `identity:get_role(role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:14.246193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:14.247609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 563/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:14 2022] GET /identity/v3/roles/dd3a79631b06401fab3c7670acb9024d => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:14.253651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:14.337761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:14.358422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cd920-fc20-40ac-b13f-5e9fb356c7a3 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:14.392041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:14.392951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:14.393235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:14.393514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] PATH_INFO: `/v3/users/81a9d1695e364eefa073b768cfb7a775` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:14.408966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] RBAC: Authorizing `identity:get_user(user_id=81a9d1695e364eefa073b768cfb7a775)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:14.410429 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:14.413033 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 551/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:14 2022] GET /identity/v3/users/81a9d1695e364eefa073b768cfb7a775 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:14.418855 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:14.502797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:14.525447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c519e-78b3-418c-afe5-51223f1a515f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:14.569979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:14.570642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:14.570929 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:14.571207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:14.641683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:14.643159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:14.732313 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:14.753474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 564/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:14.760378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:14.862264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec4f00-809c-41bc-a378-5de144f4c9e0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:14.898800 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:14.899668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:14.899952 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:14.900230 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:14.901443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:14.902869 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a89d-6ba4-4553-a68c-88db2f4fe424 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:14.904609 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 552/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:14 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:14.910927 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:15.024682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bdea0-9337-430d-8c84-4eb6cb88f794 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:15.061162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:15.061810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:15.062092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:15.062366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:15.063190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=6e5bf08e446647c3a3d14381824dd064, user.id=81a9d1695e364eefa073b768cfb7a775) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:15.063499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:15.064979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a03c3-22c3-4e69-aa52-96f4ff3adad9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:15.070651 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 565/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:14 2022] GET /identity/v3/role_assignments?user.id=81a9d1695e364eefa073b768cfb7a775&scope.project.id=6e5bf08e446647c3a3d14381824dd064&role.id=dd3a79631b06401fab3c7670acb9024d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:17.292902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064532-5027-4be1-9822-d4629897a67c None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:17.293445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064532-5027-4be1-9822-d4629897a67c None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:17.293445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0064532-5027-4be1-9822-d4629897a67c None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:17.294828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 553/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:17.300498 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:17.300921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:17.301069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:17.320167 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:17.321466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:17.361611 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:17.390820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 566/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:17.414082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0064532-5027-4be1-9822-d4629897a67c None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:17.425361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:17.521347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0064532-5027-4be1-9822-d4629897a67c None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:17.556248 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:17.557042 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:17.557328 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:17.557606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] PATH_INFO: `/v3/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:17.559089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] RBAC: Authorizing `identity:get_role(role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:17.560487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:17.561875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 554/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:17 2022] GET /identity/v3/roles/dd3a79631b06401fab3c7670acb9024d => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:17.568764 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:17.670054 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979276d5-89fd-4c99-a069-23fd1d612109 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:17.706475 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:17.707347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:17.707627 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:17.707957 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] PATH_INFO: `/v3/users/81a9d1695e364eefa073b768cfb7a775` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:17.709176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] RBAC: Authorizing `identity:get_user(user_id=81a9d1695e364eefa073b768cfb7a775)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:17.710614 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:17.712249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 567/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:17 2022] GET /identity/v3/users/81a9d1695e364eefa073b768cfb7a775 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:17.718089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:17.797095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:17.821118 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f15da8-ca5e-4086-a668-97c4fd5d24d7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:17.855931 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:17.856819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:17.857108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:17.857386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:17.858564 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:17.860001 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c63c35-ee91-4be3-8f3f-d5e03d77ee27 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:17.861754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 555/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:17 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:17.868364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:17.913258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:17.971776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9a18f-78af-429c-ab3c-34e5877cec40 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:18.007832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:18.008753 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:18.009037 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:18.009315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064/users/81a9d1695e364eefa073b768cfb7a775/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:18.011519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] RBAC: Authorizing `identity:create_grant(project_id=6e5bf08e446647c3a3d14381824dd064, user_id=81a9d1695e364eefa073b768cfb7a775, role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:18.013068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2982bf-5667-4827-9282-8d5976f54baa None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:18.026096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 568/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:58:17 2022] PUT /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064/users/81a9d1695e364eefa073b768cfb7a775/roles/dd3a79631b06401fab3c7670acb9024d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:58:20.272245 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b198f-6fbd-426c-9f39-78b12de900b1 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:20.272832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b198f-6fbd-426c-9f39-78b12de900b1 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:20.272832 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62b198f-6fbd-426c-9f39-78b12de900b1 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:20.274231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 556/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:20.279605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:20.280074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:20.280159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:20.297741 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:20.299180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:20.367843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 569/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:20.391310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b198f-6fbd-426c-9f39-78b12de900b1 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:20.495184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62b198f-6fbd-426c-9f39-78b12de900b1 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:20.530609 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:20.531412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:20.531702 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:20.531980 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] PATH_INFO: `/v3/roles/dd3a79631b06401fab3c7670acb9024d` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:20.533537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] RBAC: Authorizing `identity:get_role(role_id=dd3a79631b06401fab3c7670acb9024d)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:20.534915 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:20.536877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 557/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:20 2022] GET /identity/v3/roles/dd3a79631b06401fab3c7670acb9024d => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:20.543149 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:20.574035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:20.649087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f261ba-fcd7-4d2a-b295-da4e76d7cc71 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:20.684427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:20.685295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:20.685580 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:20.685852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] PATH_INFO: `/v3/users/81a9d1695e364eefa073b768cfb7a775` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:20.687035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] RBAC: Authorizing `identity:get_user(user_id=81a9d1695e364eefa073b768cfb7a775)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:20.688509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:20.690156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 570/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:20 2022] GET /identity/v3/users/81a9d1695e364eefa073b768cfb7a775 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:20.696240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:20.712393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:20.801647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf9207-4bb2-45cb-87e6-66407f9b7144 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:20.837048 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:20.837698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:20.837986 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:20.838265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:20.916574 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:20.918044 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:20.932774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:21.021837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 558/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:21.027938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:21.128447 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:21.132238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d850d-7718-429d-a405-8316fa13e18a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:21.167310 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:21.168190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:21.168507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:21.168787 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] PATH_INFO: `/v3/projects/6e5bf08e446647c3a3d14381824dd064` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:21.170035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] RBAC: Authorizing `identity:get_project(project_id=6e5bf08e446647c3a3d14381824dd064)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:21.171488 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cc976-ad86-4453-b523-660ab56ae5ea None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:21.173285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 571/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:21 2022] GET /identity/v3/projects/6e5bf08e446647c3a3d14381824dd064 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:21.179981 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:21.282124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562534bc-769e-4095-ace0-4142417e4f69 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:21.297350 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:21.320383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:21.321080 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:21.321368 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:21.321645 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:21.322500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] RBAC: Adding query filter params (role.id=dd3a79631b06401fab3c7670acb9024d, scope.project.id=6e5bf08e446647c3a3d14381824dd064, user.id=81a9d1695e364eefa073b768cfb7a775) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:21.322812 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:21.324325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0d5bc-6263-48de-8c6e-d8109180bc0d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:21.331127 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 559/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:21 2022] GET /identity/v3/role_assignments?user.id=81a9d1695e364eefa073b768cfb7a775&scope.project.id=6e5bf08e446647c3a3d14381824dd064&role.id=dd3a79631b06401fab3c7670acb9024d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:23.643052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464d3c-a2ef-4c42-b5d3-5a0b378c2f8d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:23.643601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464d3c-a2ef-4c42-b5d3-5a0b378c2f8d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:23.643601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464d3c-a2ef-4c42-b5d3-5a0b378c2f8d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:23.645033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 572/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:23.651068 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:23.651524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:23.651622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:23.675896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:23.677822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:23.747705 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 560/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:23.772678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9464d3c-a2ef-4c42-b5d3-5a0b378c2f8d None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:23.875643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9464d3c-a2ef-4c42-b5d3-5a0b378c2f8d None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:23.910959 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:23.911825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:23.912111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:23.912393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:23.913605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:23.915052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1da2f0-e3ef-4061-a9e2-a95424ec0253 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:23.916724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 573/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:23.922570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:24.026430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64465d78-b14d-4fd9-bbb7-0569d8016fab None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:24.034640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:24.062681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:24.063360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:24.063648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:24.063927 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:24.064682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:24.066122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9735f45-ac3e-4a75-b194-6d2ac91c2719 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:24.093360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 561/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:58:23 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:26.421284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1462-d4ba-4892-8456-49fbe88da44a None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:26.421821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1462-d4ba-4892-8456-49fbe88da44a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:26.421821 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1462-d4ba-4892-8456-49fbe88da44a None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:26.423207 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 574/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:26.429240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:26.429782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:26.429881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:26.447638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:26.448887 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:26.525485 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 562/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:26.550713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d1462-d4ba-4892-8456-49fbe88da44a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:26.590021 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:26.669749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d1462-d4ba-4892-8456-49fbe88da44a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:26.706722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:26.707422 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:26.707775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:26.708063 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:26.709357 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:26.710794 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d27c2-6f45-49e9-a3fe-0b0fd580bf30 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:26.712456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 575/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:58:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:26.718239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:26.808802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:26.822386 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef261715-2cff-479c-b94b-2434b89f8899 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:26.857207 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:26.857885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:26.858175 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:26.858455 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:26.859165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:26.860642 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29131efa-d37d-4900-a441-a6740994a61d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:26.883239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 563/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:58:26 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:29.252582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e790c-1725-4965-9385-dd21442f466b None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:29.253136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e790c-1725-4965-9385-dd21442f466b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:29.253136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e790c-1725-4965-9385-dd21442f466b None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:29.254505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 576/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:29.260223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:29.260689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:29.260830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:29.278442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:29.279649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:29.360052 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 564/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:29.384035 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e790c-1725-4965-9385-dd21442f466b None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:29.487216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e790c-1725-4965-9385-dd21442f466b None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:29.490311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:29.523838 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:29.524672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:29.524962 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:29.525242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:29.535259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:29.543778 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:29.544842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:29.545934 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 577/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:29.552148 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:29.636712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:29.668309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e92b3f-74a1-42db-bc3e-0424584ebab3 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:29.703101 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:29.703769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:29.704057 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:29.704333 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:29.705119 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:29.710821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:29.712242 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:29.721989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 565/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:29.728591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:29.843847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037611b7-2525-454a-b87b-4344fb605db9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:29.878944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:29.880294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:29.880612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:29.880896 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] PATH_INFO: `/v3/users/a1bf060eded74e74a7f514fd11878901` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:29.894633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf060eded74e74a7f514fd11878901)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:29.896849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:29.898561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 578/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:29 2022] GET /identity/v3/users/a1bf060eded74e74a7f514fd11878901 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:29.904421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:30.014015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe91dc-fcf2-48fe-b2fe-a3833b97d984 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:30.048633 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:30.049535 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:30.049822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:30.050098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:30.053696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:30.119047 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:30.120444 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:30.245876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 566/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:30.251767 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:30.276881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:30.361215 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd815-8a57-4890-8291-14a7755df400 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:30.397003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:30.397872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:30.398162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:30.398442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] PATH_INFO: `/v3/projects/c2f8230fc41144e582d5d9d25cb68541` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:30.399647 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] RBAC: Authorizing `identity:get_project(project_id=c2f8230fc41144e582d5d9d25cb68541)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:30.401127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24090e-cc6b-4257-bb72-9abf99ed043a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:30.402844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 579/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:30 2022] GET /identity/v3/projects/c2f8230fc41144e582d5d9d25cb68541 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:30.409132 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:30.445184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:30.512771 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f23f-0a45-4b99-a4a6-f759e15b5181 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:30.557673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:30.558335 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:30.558617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:30.558892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:30.559781 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=c2f8230fc41144e582d5d9d25cb68541, user.id=a1bf060eded74e74a7f514fd11878901) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:30.560094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:30.561562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36972ec6-438e-4785-8d04-4a411e7903c2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:30.567010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 567/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:30 2022] GET /identity/v3/role_assignments?user.id=a1bf060eded74e74a7f514fd11878901&scope.project.id=c2f8230fc41144e582d5d9d25cb68541&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:32.819245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cdf2-b781-49f5-9706-0b300fc2b10e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:32.819760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cdf2-b781-49f5-9706-0b300fc2b10e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:32.819760 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3cdf2-b781-49f5-9706-0b300fc2b10e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:32.821133 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 580/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:32.826408 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:32.826783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:32.826971 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:32.844636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:32.845836 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:32.915577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 568/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:32.938943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3cdf2-b781-49f5-9706-0b300fc2b10e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:33.032458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:33.044326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3cdf2-b781-49f5-9706-0b300fc2b10e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:33.080214 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:33.080956 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.081242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.081514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.091734 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:33.100223 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:33.101193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:33.102065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 581/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:33.107649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:33.138027 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:33.210677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbc25c-6904-4407-a713-b93c76e98cea None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:33.246536 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:33.247494 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.247899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.248299 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.249439 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:33.249882 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:33.252287 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:33.262608 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 569/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:33.268572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:33.375023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e92470-fb8a-4b31-aabd-c4cf3f1d95c9 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:33.409057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:33.409959 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.410247 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.410526 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] PATH_INFO: `/v3/users/a1bf060eded74e74a7f514fd11878901` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.412431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf060eded74e74a7f514fd11878901)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:33.413933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:33.415652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 582/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:33 2022] GET /identity/v3/users/a1bf060eded74e74a7f514fd11878901 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:33.421515 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:33.514828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:33.524969 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120322d8-38d2-4a6e-8a74-05cc63c93c07 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:33.559362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:33.560220 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.560539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.560819 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] PATH_INFO: `/v3/projects/c2f8230fc41144e582d5d9d25cb68541` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.562036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] RBAC: Authorizing `identity:get_project(project_id=c2f8230fc41144e582d5d9d25cb68541)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:33.563510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99bc76-852f-4697-94b8-f1ec5669ae8d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:33.565350 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 570/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:33 2022] GET /identity/v3/projects/c2f8230fc41144e582d5d9d25cb68541 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:33.572393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:33.592818 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:33.686714 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96877-4443-4f96-a6cb-f9dbdf48ecc2 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:33.726110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:33.727029 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.727312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.727590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] PATH_INFO: `/v3/projects/c2f8230fc41144e582d5d9d25cb68541/users/a1bf060eded74e74a7f514fd11878901/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.729613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2f8230fc41144e582d5d9d25cb68541, user_id=a1bf060eded74e74a7f514fd11878901, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:33.731069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aef9d4-a7d5-4aef-8a0a-c965b7b8dfd2 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:33.745804 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 583/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:58:33 2022] PUT /identity/v3/projects/c2f8230fc41144e582d5d9d25cb68541/users/a1bf060eded74e74a7f514fd11878901/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:58:36.019184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8c054-9568-4981-b7d3-5a7e19f4d551 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.019715 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8c054-9568-4981-b7d3-5a7e19f4d551 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.019825 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8c054-9568-4981-b7d3-5a7e19f4d551 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.021739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 571/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:36.027419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.027944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.028034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.045995 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:36.047190 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:36.124751 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 584/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:36.148811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8c054-9568-4981-b7d3-5a7e19f4d551 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:36.253468 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8c054-9568-4981-b7d3-5a7e19f4d551 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:36.290871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:36.291587 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.291871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.292145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.304412 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:36.312100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:36.313253 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:36.314678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 572/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:36.320284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:36.335465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:36.435660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb981e-3979-4ea9-8b1c-d59bd12161ef None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:36.470784 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:36.471451 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.471733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.472007 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.472776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:36.473083 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:36.474509 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:36.482713 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 585/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:36.488699 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:36.512736 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:36.592845 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3e624-75b1-4b94-890e-dc20daa9c559 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:36.626860 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:36.627733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.628017 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.628296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] PATH_INFO: `/v3/users/a1bf060eded74e74a7f514fd11878901` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.630234 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=a1bf060eded74e74a7f514fd11878901)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:36.631678 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:36.633355 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 573/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:36 2022] GET /identity/v3/users/a1bf060eded74e74a7f514fd11878901 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:36.639711 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:36.717065 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:36.750663 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7503623-29c4-4a32-b955-4491cf59ca82 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:36.784991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:36.785824 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:36.785950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:36.786222 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:36.855497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:36.856914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:36.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:36.958747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 586/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:36.964711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:37.053165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:37.066400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd85a7-e5b6-4205-a2b5-8497069ad6e6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:37.100511 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:37.101372 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.101649 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.101918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] PATH_INFO: `/v3/projects/c2f8230fc41144e582d5d9d25cb68541` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.103087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] RBAC: Authorizing `identity:get_project(project_id=c2f8230fc41144e582d5d9d25cb68541)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:37.104552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29badf6d-efd9-4a0d-bcfe-c4eb13e0de03 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:37.106396 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 574/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:36 2022] GET /identity/v3/projects/c2f8230fc41144e582d5d9d25cb68541 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:37.112686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:37.213403 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5122e-4e9d-4805-87b9-4c0ad956841e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:37.248159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:37.248841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.249126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.249497 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.250235 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=c2f8230fc41144e582d5d9d25cb68541, user.id=a1bf060eded74e74a7f514fd11878901) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:37.250546 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:37.251987 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708f6ce-c6e7-4782-ace5-c9e9823e3405 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:37.258938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 587/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:37 2022] GET /identity/v3/role_assignments?user.id=a1bf060eded74e74a7f514fd11878901&scope.project.id=c2f8230fc41144e582d5d9d25cb68541&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:39.585139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e3f9-53b9-4f6e-8b17-d33b810c3a46 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:39.585681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e3f9-53b9-4f6e-8b17-d33b810c3a46 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:39.585681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1e3f9-53b9-4f6e-8b17-d33b810c3a46 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:39.587124 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 575/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:39.592661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:39.593092 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:39.593221 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:39.610532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:39.611746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:39.648299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:39.696242 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 588/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:39.719185 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1e3f9-53b9-4f6e-8b17-d33b810c3a46 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:39.825130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1e3f9-53b9-4f6e-8b17-d33b810c3a46 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:39.861663 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:39.862313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:39.862599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:39.862876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:39.889554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:39.942600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:39.944093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:40.066323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 576/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:40.072609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:40.188527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ccd0c1-cd92-4494-a513-b4bb4fc07e92 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:40.227371 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:40.228166 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:40.228487 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:40.228770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:40.235419 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:40.236912 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-b79dfe45-8663-4061-92b2-079ee4625736 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:40.260696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:40.261952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:40.263138 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 589/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 19:58:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:40.268043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:40.327523 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:40.373303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0f858-c21d-4754-aeb4-83194a32b20e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:40.408303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:40.409013 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:40.409303 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:40.409584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:40.410318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:40.410668 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:40.412054 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984039e3-5c74-45db-9537-282df33025e9 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:40.419446 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 577/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 19:58:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:42.719950 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ddca-cce6-4c31-a7a5-c88552cf9cf3 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.720530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ddca-cce6-4c31-a7a5-c88552cf9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.720530 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233ddca-cce6-4c31-a7a5-c88552cf9cf3 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.721915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 590/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:42.727775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d2cc-8ef3-441b-ac7a-1596cb91dada None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.728194 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d2cc-8ef3-441b-ac7a-1596cb91dada None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.728349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07d2cc-8ef3-441b-ac7a-1596cb91dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.746988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-0a07d2cc-8ef3-441b-ac7a-1596cb91dada None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:42.748218 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0a07d2cc-8ef3-441b-ac7a-1596cb91dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:42.817769 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 578/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:42.842683 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233ddca-cce6-4c31-a7a5-c88552cf9cf3 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:42.871828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:42.951682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233ddca-cce6-4c31-a7a5-c88552cf9cf3 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:42.985504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:42.986161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.986444 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.986720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.987457 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:42.988866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf5034-7c9f-4c03-ac4c-e3f91783ff93 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:43.005078 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 591/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu May 19 19:58:42 2022] POST /identity/v3/domains => generated 267 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:45.311460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612538-f5cf-460e-8f62-08beda554715 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.312013 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612538-f5cf-460e-8f62-08beda554715 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.312013 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612538-f5cf-460e-8f62-08beda554715 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.313475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 579/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:45.318952 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.319395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.319540 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.340520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:45.340520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:45.409330 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 592/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:45.433258 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e612538-f5cf-460e-8f62-08beda554715 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:45.485951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:45.540165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e612538-f5cf-460e-8f62-08beda554715 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:45.576184 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:45.577164 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.577443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.577722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] PATH_INFO: `/v3/domains/62b5fe7a64b840f4a5745ef77933d9c4` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.588667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62b5fe7a64b840f4a5745ef77933d9c4)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:45.590117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39f97-a686-4d01-bcd9-a87b3f22eef7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:45.591773 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 580/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 19:58:45 2022] GET /identity/v3/domains/62b5fe7a64b840f4a5745ef77933d9c4 => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:45.598128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:45.614308 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:45.717153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b11c99-3cd2-4f80-a015-8fdcc8604115 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:45.760762 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:45.761445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.761775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.762052 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.762791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:45.764258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42085f81-178e-4ae1-bd80-88203f15284a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:45.790515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 593/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 19:58:45 2022] POST /identity/v3/projects => generated 361 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:48.112827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163cb6-89df-4ea0-89bd-363435617eba None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:48.113389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163cb6-89df-4ea0-89bd-363435617eba None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:48.113389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163cb6-89df-4ea0-89bd-363435617eba None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:48.114749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 581/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:48.121402 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:48.121892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:48.121988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:48.139820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:48.141043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:48.201862 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:48.210524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 594/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:48.235360 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163cb6-89df-4ea0-89bd-363435617eba None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:48.355094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163cb6-89df-4ea0-89bd-363435617eba None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:48.367734 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:48.391752 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:48.392651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:48.392945 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:48.393229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] PATH_INFO: `/v3/domains/62b5fe7a64b840f4a5745ef77933d9c4` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:48.394422 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] RBAC: Authorizing `identity:get_domain(domain_id=62b5fe7a64b840f4a5745ef77933d9c4)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:48.395886 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9004960-4dc4-45b5-8e6e-36de160cfcda None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:48.398229 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 582/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 19:58:48 2022] GET /identity/v3/domains/62b5fe7a64b840f4a5745ef77933d9c4 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:48.403994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:48.504155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87296fdd-c998-43b1-9820-038cd0fd805a None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:48.547314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:48.548001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:48.548284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:48.548600 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:48.549583 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:48.551034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcf4d2-9a91-455a-91ce-5594eabefea9 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:48.571916 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 595/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:58:48 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:58:50.907582 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c46d-e530-4cec-9b5a-439c61c02014 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:50.908113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c46d-e530-4cec-9b5a-439c61c02014 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:50.908223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9575c46d-e530-4cec-9b5a-439c61c02014 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:50.909572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 583/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:50.915970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:50.916490 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:50.916587 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:50.933898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:50.935094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:50.983479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:51.003696 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 596/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:51.028808 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575c46d-e530-4cec-9b5a-439c61c02014 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:51.133211 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9575c46d-e530-4cec-9b5a-439c61c02014 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:51.167830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:51.168583 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:51.168875 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:51.169155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:51.180896 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:51.188763 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:51.189834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:51.190802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 584/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:51.197216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:51.314604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce64e3a-8e9e-4eda-87e5-28e6783028bb None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:51.351485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:51.352156 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:51.352474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:51.352759 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:51.353514 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:51.353827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:51.355219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:51.363457 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 597/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:58:51.370088 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:51.411272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:51.476914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45c820-44ff-4e41-8573-b1e1133afad0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:51.516947 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:51.517847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:51.518134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:51.518413 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] PATH_INFO: `/v3/users/091f39d608254e9b93b81c198e0a26c9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:51.532298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] RBAC: Authorizing `identity:get_user(user_id=091f39d608254e9b93b81c198e0a26c9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:51.533779 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:51.535471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 585/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:51 2022] GET /identity/v3/users/091f39d608254e9b93b81c198e0a26c9 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:51.541844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:51.589412 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:51.655194 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa252c55-f31e-477e-aef2-781c40977b49 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:51.690110 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:51.690770 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:51.691058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:51.691339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:51.766673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:51.768064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:51.819068 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:51.877180 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 598/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:51.883598 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:51.987553 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:51.994467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1721-a30e-4182-b61c-e79f894611b7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:52.032605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:52.033384 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:52.033674 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:52.033954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] PATH_INFO: `/v3/projects/4c1268876d7e4884a63686ae044969e0` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:52.035134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] RBAC: Authorizing `identity:get_project(project_id=4c1268876d7e4884a63686ae044969e0)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:52.036601 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389d42f-d2f8-4b2a-aebe-df87b2ff5b9e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:52.038923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 586/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:51 2022] GET /identity/v3/projects/4c1268876d7e4884a63686ae044969e0 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:52.045044 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:52.163974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8dae-496d-4846-befe-6fd93a481d02 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:52.200284 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:52.200976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:52.201264 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:52.201544 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:52.202386 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=4c1268876d7e4884a63686ae044969e0, user.id=091f39d608254e9b93b81c198e0a26c9) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:52.202703 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:52.204149 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbad3ff-ad1c-4e30-80fd-a0fb51941c3f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:52.209761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 599/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:52 2022] GET /identity/v3/role_assignments?user.id=091f39d608254e9b93b81c198e0a26c9&scope.project.id=4c1268876d7e4884a63686ae044969e0&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:54.517938 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff367c-9f11-4b08-843a-23ec9416b6d9 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:54.518319 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff367c-9f11-4b08-843a-23ec9416b6d9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:54.518510 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff367c-9f11-4b08-843a-23ec9416b6d9 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:54.519904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 587/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:54.526022 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:54.526454 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:54.526582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:54.543832 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:54.545041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:54.546991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:54.614276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 600/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:54.636999 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff367c-9f11-4b08-843a-23ec9416b6d9 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:54.749815 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff367c-9f11-4b08-843a-23ec9416b6d9 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:54.786672 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:54.787577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:54.787948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:54.788227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:54.798423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:54.806421 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:54.807471 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 19:58:54.808503 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 588/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:54.813872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:54.915677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c40f43-3cdd-44be-bb63-13582b15dfd2 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:54.951075 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:54.951743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:54.952025 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:54.952299 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:54.953079 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:54.953384 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:54.954756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:54.960248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:54.964828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 601/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:54.970544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:54.997409 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:55.074754 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc9d0c-fdf0-4da9-b5af-fe61eef9fa2e None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:55.109027 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:55.109902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:55.110195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:55.110477 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] PATH_INFO: `/v3/users/091f39d608254e9b93b81c198e0a26c9` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:55.111677 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] RBAC: Authorizing `identity:get_user(user_id=091f39d608254e9b93b81c198e0a26c9)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:55.113171 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:55.114830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 589/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:54 2022] GET /identity/v3/users/091f39d608254e9b93b81c198e0a26c9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:55.122139 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:55.223111 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a4a4-22e1-423c-96cd-ef615a4c997b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:55.257888 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:55.258744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:55.259031 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:55.259389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] PATH_INFO: `/v3/projects/4c1268876d7e4884a63686ae044969e0` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:55.260661 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1268876d7e4884a63686ae044969e0)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:55.262023 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6e366-47fd-4077-81a7-f8d60c04b083 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:55.263724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 602/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:55 2022] GET /identity/v3/projects/4c1268876d7e4884a63686ae044969e0 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:55.269647 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:55.366370 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:55.375527 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347851d4-84cc-4f8e-a0dc-d24ed8ce45a5 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:55.409902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:55.410790 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:55.411070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:55.411341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] PATH_INFO: `/v3/projects/4c1268876d7e4884a63686ae044969e0/users/091f39d608254e9b93b81c198e0a26c9/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:55.414117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c1268876d7e4884a63686ae044969e0, user_id=091f39d608254e9b93b81c198e0a26c9, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:55.415554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ed36-c65b-41d7-a05a-77514d790e55 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:55.429210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 590/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 19:58:55 2022] PUT /identity/v3/projects/4c1268876d7e4884a63686ae044969e0/users/091f39d608254e9b93b81c198e0a26c9/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:58:57.673960 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340018-2212-4796-8b1d-07c3faebfb3f None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:57.674766 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340018-2212-4796-8b1d-07c3faebfb3f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:57.674872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc340018-2212-4796-8b1d-07c3faebfb3f None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:57.676239 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 603/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:57.682358 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:57.682827 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:57.682942 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:57.700621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:57.701818 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:57.784554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 591/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:58:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:58:57.809171 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc340018-2212-4796-8b1d-07c3faebfb3f None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:57.885976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:57.939619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc340018-2212-4796-8b1d-07c3faebfb3f None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:57.975961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:57.976812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:57.977101 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:57.977379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:57.989176 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:58:57.997716 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:58:57.998756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:58:57.999898 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 604/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 19:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:58:58.005036 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:58.096041 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:58.116543 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7cdbfb-2154-409b-a4a1-8f92b03f495d None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:58.152701 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:58.153367 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:58.153651 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:58.153921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:58.154673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:58.154981 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:58.156345 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:58.165084 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 592/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 19:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:58.171070 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:58.278900 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8932c-7e40-4165-996c-cfaf20bcfd14 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:58.315887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:58.326464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:58.327350 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:58.327634 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:58.327914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] PATH_INFO: `/v3/users/091f39d608254e9b93b81c198e0a26c9` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:58.329159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] RBAC: Authorizing `identity:get_user(user_id=091f39d608254e9b93b81c198e0a26c9)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:58.330639 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:58.332298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 605/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 19:58:58 2022] GET /identity/v3/users/091f39d608254e9b93b81c198e0a26c9 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:58.338518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:58.442371 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490c92c-d365-42e8-8b31-1173f934df75 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:58.478652 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:58.479307 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:58.479595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:58.479871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:58.499149 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:58.550347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:58.551847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:58.662463 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 593/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 19:58:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:58:58.668667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:58.776276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce867a-f0e9-4be2-9c79-5897aa0c4545 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:58.811547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:58.812502 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:58.812792 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:58.813100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] PATH_INFO: `/v3/projects/4c1268876d7e4884a63686ae044969e0` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:58.814364 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] RBAC: Authorizing `identity:get_project(project_id=4c1268876d7e4884a63686ae044969e0)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:58.815828 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a5181-5dc8-4b5d-ad7e-d8a330670f82 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:58.817674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 606/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 19:58:58 2022] GET /identity/v3/projects/4c1268876d7e4884a63686ae044969e0 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:58:58.824667 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:58.871495 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:58.927352 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81646e-2490-4ece-b2a0-61a7d2735cc3 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:58.962481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:58.963157 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:58.963442 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:58.963725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:58.964605 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=4c1268876d7e4884a63686ae044969e0, user.id=091f39d608254e9b93b81c198e0a26c9) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:58:58.964923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:58.966423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e6c14-f4a2-4c64-81d1-dcf1e1fb23d7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:58.973168 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 594/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 19:58:58 2022] GET /identity/v3/role_assignments?user.id=091f39d608254e9b93b81c198e0a26c9&scope.project.id=4c1268876d7e4884a63686ae044969e0&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:59:08.806744 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=68 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:59:08.970157 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=46 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:59:59.213298 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e040ec-009e-4237-8e0f-8f2ff6292464 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:59:59.213872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e040ec-009e-4237-8e0f-8f2ff6292464 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:59:59.213872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e040ec-009e-4237-8e0f-8f2ff6292464 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:59:59.215456 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 607/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 19:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:59:59.220609 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:59:59.221061 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:59:59.221157 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:59:59.238877 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:59:59.240163 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:59:59.322077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 595/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 19:59:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:59:59.347339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e040ec-009e-4237-8e0f-8f2ff6292464 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:59:59.464776 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e040ec-009e-4237-8e0f-8f2ff6292464 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:59:59.503574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:59:59.509829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:59:59.509829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:59:59.509829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:59:59.522345 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:59:59.523889 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:59:59.544084 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:59:59.545295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 19:59:59.546174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:59:59.546174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:59:59.546174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:59:59.546174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 19:59:59.546174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 608/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 19:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:59:59.552776 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:59:59.682791 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cefa33-3067-44a4-a714-ba5f84a876c0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:59:59.721150 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:59:59.730385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:59:59.731029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:59:59.731313 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:59:59.731589 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:59:59.732317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:59:59.732665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:59:59.734089 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d297c-dd26-45dc-8f17-151e9a185ae8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:59:59.751093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 596/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 19:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:59:59.752296 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:59:59.819691 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:59:59.874055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f713e20a-f6e0-496b-b742-560c43fec48a None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:59:59.914127 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:59:59.915131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:59:59.915420 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:59:59.915795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:59:59.916562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:59:59.918056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8dd99-f13f-48ba-a3fd-3d442c407abb None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:59:59.946455 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 609/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 19:59:59 2022] POST /identity/v3/users => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:00:02.640972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9d964-015c-4723-80eb-524e63640885 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:02.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9d964-015c-4723-80eb-524e63640885 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:02.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9d964-015c-4723-80eb-524e63640885 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:02.643095 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 597/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:02.650053 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:02.650597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:02.650597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:02.672043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:02.673375 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:02.772567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 610/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:02.798636 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9d964-015c-4723-80eb-524e63640885 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:02.947282 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9d964-015c-4723-80eb-524e63640885 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:03.006538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:03.008060 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:03.008504 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:03.009021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:03.025572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:03.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:03.035382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:03.037182 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 598/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:00:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:03.042869 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:03.093892 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:03.174999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7afef68-8fbd-4953-9437-b43d7ec2787b None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:03.218567 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:03.219534 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:03.219866 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:03.220276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:03.221295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:03.221702 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:03.223991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:03.234973 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 611/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:03.241198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:03.318923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:03.385186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2d34b-9569-43a4-80fa-88ae17fd7842 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:03.423381 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:03.424187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:03.424509 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:03.425037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:03.436966 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:03.438430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:03.461043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:03.462224 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:03.465078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:03.465078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:03.465078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:03.465078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:03.465078 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 599/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:00:03.469103 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:03.603383 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:03.607342 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08a495-a1c1-4fed-a97a-fa1c571b4817 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:03.652216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:03.655210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:03.655210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:03.655210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:03.655210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:03.655210 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:03.655928 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:03.672450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 612/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:03.677487 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:03.805126 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ee45a-b2ec-4c94-81e8-58f7c65b717f None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:03.806920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:03.850103 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:03.850928 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:03.851183 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:03.851462 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] PATH_INFO: `/v3/users/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:03.861110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:03.862863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:03.879032 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:03.880953 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:03.882049 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:03.884584 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 600/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:03.887148 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:04.011558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5c3ee-9182-4d79-9ad2-00fffcda3173 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:04.053366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:04.053971 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:04.054258 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:04.054536 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:04.055393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:04.055704 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:04.057193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:04.076505 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 613/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:04.083526 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:04.205347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5596161-c214-4ee6-bffd-adb4401e7f2b None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:04.251365 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:04.252029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:04.252314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:04.252622 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:04.286382 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:04.337443 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:04.339404 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:04.476209 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 601/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:00:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:00:04.484339 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:04.519612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:04.628248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ee53-5af1-4f50-b6a4-9945c7123ece None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:04.672503 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:04.673393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:04.673674 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:04.673958 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:04.686573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:04.688351 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:04.715379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:04.716474 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:04.725081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:04.725081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:04.725081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:04.725081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:04.725081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 614/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:04.725451 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:04.803692 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:04.860606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090a349-86be-4a66-98a6-6d15153d1f3d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:04.905331 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:04.906022 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:04.906423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:04.906826 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:04.907682 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:04.908125 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:04.909775 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:04.925640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 602/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:04.930660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:05.020594 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:05.070874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc8955-b3f5-4d83-bf32-09513977b11b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:05.121868 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:05.122680 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:05.123013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:05.123257 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:05.136034 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:05.137468 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:05.168441 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:05.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:05.170343 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:05.176524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:05.176816 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 615/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:05.315239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed30291-ff8d-45f6-95ed-42d00faf1b53 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:05.346385 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:05.362690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:05.363447 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:05.363817 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:05.364113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:05.364948 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:05.365300 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:05.366923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286248e-f7d1-4dcd-9329-6ec11ff7acb8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:05.384454 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 603/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:00:05.389510 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:05.541178 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cf2da-ef30-432d-a0a9-cd65257d7a96 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:05.579118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:05.587057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:05.587859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:05.588245 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:05.588672 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:05.589410 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=d1990383854d4f97b2d2f4bdd53a05f8) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:05.589756 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:05.591448 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0d340-3c2c-429c-8cbd-3466e069f2d0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:05.608882 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 616/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:00:05 2022] GET /identity/v3/role_assignments?user.id=d1990383854d4f97b2d2f4bdd53a05f8&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:07.956562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32866c35-8602-4d67-b864-834eb0a32fbb None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:07.957100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32866c35-8602-4d67-b864-834eb0a32fbb None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:07.957100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32866c35-8602-4d67-b864-834eb0a32fbb None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:07.959525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 604/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:07.966218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:07.966752 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:07.966859 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:07.985312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:07.992326 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:08.090390 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 617/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:08.116524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32866c35-8602-4d67-b864-834eb0a32fbb None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:08.262821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32866c35-8602-4d67-b864-834eb0a32fbb None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:08.302723 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:08.303729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:08.304043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:08.304323 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:08.318539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:08.328016 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:08.329081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:08.329923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 605/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:08.336994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:08.452321 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a405e5-2837-47c2-893f-5868e2b598fd None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:08.481524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:08.497043 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:08.497990 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:08.498445 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:08.498856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:08.500038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] RBAC: Adding query filter params (name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:08.500572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:08.502802 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:08.515442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 618/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:08.522479 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:08.583696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:08.649539 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fabf9-4656-4326-9edb-cc1b4a14c7ca None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:08.699525 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:08.700681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:08.700977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:08.701265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:08.713844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:08.715435 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:08.736309 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:08.737934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:08.739077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:08.739077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:08.739077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:08.739077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:08.739077 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 606/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:00:08.743976 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:08.864265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a750a-cf96-4be6-900d-ee5f84d25a73 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:08.904227 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:08.904893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:08.905128 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:08.905421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:08.906212 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:08.906564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:08.907978 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:08.921992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 619/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:08.928173 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:09.036717 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:09.045786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7cde19-6039-4c51-8f17-bc75c10201e6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:09.083169 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:09.084010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:09.084294 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:09.084599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:09.093657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:09.095136 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:09.113003 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:09.114217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:09.115160 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 607/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:09.120954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:09.182019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:09.241489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90869590-0c08-4f16-933d-cf45729eab60 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:09.283849 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:09.284698 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:09.284988 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:09.285268 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:09.286748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:09.287061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:09.288725 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:09.309352 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 620/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:09.315393 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:09.442911 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60283905-32f5-42dc-989a-b8f46be809ab None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:09.486465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:09.487273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:09.487549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:09.487811 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:09.499277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:09.500980 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:09.524189 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:09.525425 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:09.526496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:09.526496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:09.526496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:09.526496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:09.526496 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 608/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:09.532393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:09.649338 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36e27c-ed03-470b-a7ab-5ce2c8be4032 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:09.651863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:09.695174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:09.695773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:09.696094 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:09.696471 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:09.697209 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:09.697519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:09.698909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:09.720970 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 621/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:09.728238 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:09.737982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:09.847680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e5463-cb52-4bcd-a09f-4c63183c28bf None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:09.887272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:09.888082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:09.888369 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:09.888676 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:09.896098 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:09.897712 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:09.913621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:09.914695 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:09.915538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 609/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:09.921047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:10.040283 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23028a-fc6f-4ef5-a80b-2b528119392b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:10.082421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:10.083615 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:10.083901 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:10.084191 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:10.085012 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:10.085328 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:10.086786 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235201ce-66f7-4215-9caf-773a4e2b296b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:10.104437 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 622/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:10.105638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:10.179279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:10.219210 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09580d-d502-4f7d-bbef-0fddc792ab70 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:10.261261 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:10.262166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] REQUEST_METHOD: `PUT` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:10.262586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:10.262861 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/d1990383854d4f97b2d2f4bdd53a05f8/roles/edcdc7ce16714570a4a95c51db8a0e77` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:10.286115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=d1990383854d4f97b2d2f4bdd53a05f8, role_id=edcdc7ce16714570a4a95c51db8a0e77)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:10.287863 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42d2f9-6e7c-4e05-83e9-96e7fd20638e None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:10.304364 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 610/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:00:10 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/d1990383854d4f97b2d2f4bdd53a05f8/roles/edcdc7ce16714570a4a95c51db8a0e77 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:00:12.764610 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8f20-db11-448b-b465-6167be0c9355 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:12.765136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8f20-db11-448b-b465-6167be0c9355 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:12.765136 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d8f20-db11-448b-b465-6167be0c9355 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:12.766551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 623/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:12.772743 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:12.773198 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:12.773310 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:12.796954 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:12.798187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:12.877151 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 611/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:12.903217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8f20-db11-448b-b465-6167be0c9355 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:12.933392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:13.023974 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d8f20-db11-448b-b465-6167be0c9355 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:13.064586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:13.065418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:13.065717 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:13.065991 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:13.078930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:13.088466 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:13.089551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:13.090553 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 624/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:13.099602 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:13.157402 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:13.219711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ab2d8-b85b-49d1-be82-c8f43de60378 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:13.259725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:13.260464 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:13.260749 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:13.261028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:13.261847 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] RBAC: Adding query filter params (name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:13.262156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:13.263830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:13.273884 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 612/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:13.279748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:13.376073 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:13.393847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34cb29e-43ba-421e-a8cd-6a4d267c1382 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:13.432118 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:13.432943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:13.434551 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:13.434835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:13.446789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:13.448256 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:13.470289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:13.471322 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:13.472359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:13.472359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:13.472359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:13.472359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:13.472359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 625/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:13.479923 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:13.614562 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f93dc3-92db-4e53-b7c3-826fdf824469 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:13.627161 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:13.663476 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:13.664379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:13.664828 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:13.665202 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:13.666142 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:13.666506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:13.668235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:13.684600 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 613/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:13.692183 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:13.812244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b6ffc-6c8d-44b4-b6ff-0aab9f126916 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:13.851763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:13.852688 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:13.852977 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:13.853267 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] PATH_INFO: `/v3/users/placement` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:13.856380 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:13.863117 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:13.866295 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:13.886169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:13.887193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:13.888248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 626/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:13.893880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:14.018021 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4fe68-c010-4ca0-8f81-736ea1bbf34c None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:14.061572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:14.062261 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:14.062628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:14.062901 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:14.063858 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:14.064239 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:14.065866 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:14.087450 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 614/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:14.093459 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:14.209769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166c99aa-670c-4286-802f-a056b2bee13f None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:14.250945 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:14.251637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:14.251925 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:14.252208 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:14.323829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:14.331449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:14.333489 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:14.452460 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 627/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:00:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:00:14.455423 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:14.485109 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:14.573900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128ad6b-9ae2-468e-ace1-18eccfc9bbff None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:14.619339 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:14.620137 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:14.620446 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:14.620727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:14.633467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:14.635154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:14.658517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:14.659566 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:14.660529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:14.660529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:14.660529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:14.660529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:14.665453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 615/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:14.666393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:14.779748 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:14.787837 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6bec0-d329-43f8-9942-370c96d04392 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:14.829248 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:14.830746 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:14.831049 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:14.831355 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:14.832145 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:14.833018 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:14.834531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:14.847485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 628/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:14.853791 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:14.951311 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:14.979116 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591d72c-7e4f-47dd-86b0-1d0f83e9b56d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:15.016702 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:15.017690 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:15.018081 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:15.018466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:15.027729 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:15.029361 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:15.045758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:15.046880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:15.053139 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 616/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:15.054205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:15.180113 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699410d9-82c7-4628-b847-181b04d834fd None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:15.217407 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:15.218076 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:15.218396 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:15.218677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:15.219485 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:15.219795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:15.221289 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c41a9-60ef-4a06-bd9f-c5cd5896755c None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:15.235980 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 629/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:15.242688 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:15.376902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2e426-fd39-41d5-8d98-51c0d5800ec0 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:15.426344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:15.427165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:15.427518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:15.427870 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:15.428908 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] RBAC: Adding query filter params (role.id=edcdc7ce16714570a4a95c51db8a0e77, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=d1990383854d4f97b2d2f4bdd53a05f8) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:15.429222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:15.430983 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caf49a-701c-4ead-90d7-43bca56f81b2 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:15.439537 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 617/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:00:15 2022] GET /identity/v3/role_assignments?user.id=d1990383854d4f97b2d2f4bdd53a05f8&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=edcdc7ce16714570a4a95c51db8a0e77 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:00:17.863925 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf6e-cdec-45d4-80cf-efb7e9a802e4 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:17.864597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf6e-cdec-45d4-80cf-efb7e9a802e4 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:17.864597 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbf6e-cdec-45d4-80cf-efb7e9a802e4 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:17.866152 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 630/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:17.871902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:17.872318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:17.872639 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:17.877964 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:17.896742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:17.898856 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:17.987762 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 618/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:00:18.013643 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbf6e-cdec-45d4-80cf-efb7e9a802e4 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:18.033591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:18.147093 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbf6e-cdec-45d4-80cf-efb7e9a802e4 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:18.200484 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:18.201365 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:18.201670 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:18.201989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:18.213601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:18.221747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:18.223381 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:18.225188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 631/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:18.230005 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:18.349383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d36191-9583-4f98-9cd7-7aab85cc63fb None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:18.386520 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:18.392547 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:18.393324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:18.393617 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:18.393941 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:18.394814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:18.395129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:18.396694 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:18.419794 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 619/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:18.421598 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:18.517003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:18.546879 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a0e26-bd70-41be-b1fd-2fb942410927 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:18.600872 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:18.601673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:18.601963 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:18.602294 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:18.615636 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:18.617285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:18.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:18.640947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:18.641808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:18.641808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:18.641808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:18.641808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:18.641808 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 632/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:18.648555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:18.774739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be36c94-3603-43a8-acad-3d76e9512d60 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:18.819890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:18.820978 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:18.821648 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:18.822262 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:18.823581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:18.824216 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:18.826588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:18.842199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 620/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:18.849375 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:18.973678 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302674-5fb8-485d-8d46-c9016a19c3a0 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:19.007481 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:19.015612 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:19.016561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:19.016909 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:19.017270 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:19.026393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:19.027874 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:19.046119 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:19.047461 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:19.048796 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 633/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:19.055203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:19.124603 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:19.177742 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78af18-66e4-4c65-8244-5471bbcfa8c7 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:19.217557 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:19.218263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:19.218555 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:19.218840 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:19.219738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:19.220056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:19.221758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:19.242803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 621/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:19.249100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:19.376126 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c81dda6-d3c9-4b3d-a83d-425cd53f6238 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:19.420345 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:19.421137 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:19.421427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:19.421728 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:19.498162 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:19.516894 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:19.518582 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:19.643903 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 634/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:00:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:00:19.645822 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:19.768881 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9ae5-c448-416e-965d-b34092cf0a93 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:19.785788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:19.812143 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:19.813006 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:19.813294 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:19.813579 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:19.826573 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:19.828083 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:19.849849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:19.850965 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:19.851987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:19.851987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:19.851987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:19.851987 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:19.857198 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:19.860492 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 622/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:19.967730 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:19.980985 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839d089-8a1a-4ffe-a1a3-6804e2ef70fa None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:20.028437 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:20.029051 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:20.029415 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:20.029619 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:20.030369 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:20.030844 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:20.032082 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:20.051314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 635/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:00:20.057380 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:20.200780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca8517-0df8-4a6a-8e87-49c825173735 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:20.243959 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:20.244813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:20.245223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:20.245613 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:20.254599 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:20.256390 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:20.277892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:20.278883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:20.279731 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:20.280453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 623/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:20.287188 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:20.421605 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3512e23-7b6c-4246-9183-2894aefd02d6 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:20.464379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:20.465080 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:20.465367 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:20.465653 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] PATH_INFO: `/v3/projects` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:20.466450 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:20.466768 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:20.468259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4379c-0ae4-4879-9ec4-a46e590e96c0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:20.481736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 636/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:20.487841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:20.520657 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:20.621085 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee04d41-1739-4e11-8265-cdcf1166d6c6 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:20.680231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:20.681696 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:20.682172 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:20.682581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:20.683594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=d1990383854d4f97b2d2f4bdd53a05f8) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:20.684111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:20.686774 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f937b-2eec-4177-8e57-fc8a38e6be93 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:20.698851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 624/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:00:20 2022] GET /identity/v3/role_assignments?user.id=d1990383854d4f97b2d2f4bdd53a05f8&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:22.977073 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd60c-dffa-44fd-9016-e2baa9c6c57d None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:22.977737 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd60c-dffa-44fd-9016-e2baa9c6c57d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:22.977737 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfd60c-dffa-44fd-9016-e2baa9c6c57d None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:22.979153 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 637/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:22.985012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:22.985632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:22.985632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.003298 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:23.005188 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:23.088449 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 625/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:23.112430 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfd60c-dffa-44fd-9016-e2baa9c6c57d None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:23.126087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:23.224967 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfd60c-dffa-44fd-9016-e2baa9c6c57d None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:23.260645 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:23.261356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:23.261641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:23.261921 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.272314 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:23.279999 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:23.281058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:23.282087 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 638/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:23.287910 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:23.308154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:23.391336 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127e4b1-a2e3-4912-8ba8-4f2882a71f44 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:23.435815 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:23.436514 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:23.436807 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:23.437087 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] PATH_INFO: `/v3/roles` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.437837 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:23.438156 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:23.439521 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:23.447841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 626/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:23.453933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:23.522687 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:23.559238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b06d9f-c496-4980-a6b0-7c8899919db3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:23.592739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:23.593523 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:23.593810 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:23.594086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.605788 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:23.607335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:23.626169 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:23.627197 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:23.628230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:23.628230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:23.628230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:23.628230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:23.628230 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 639/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:23.633552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:23.725844 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:23.745920 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f815-0425-4d6b-92a8-ffb4db539490 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:23.781365 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:23.781940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:23.782231 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:23.782512 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.783235 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:23.783549 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:23.784958 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:23.798028 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 627/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:23.804382 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:23.907334 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535d2e-a5a4-46b6-b651-aace0fc9401d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:23.929158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:23.943829 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:23.944660 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:23.944944 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:23.945218 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] PATH_INFO: `/v3/users/placement` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:23.953552 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:23.955002 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:23.969427 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:23.970532 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:23.971624 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 640/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:23.977277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.088930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065caa28-a7fe-4928-9c53-0758df07ccc8 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.125940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.126640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.126933 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.127217 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] PATH_INFO: `/v3/users` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.128082 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:24.128427 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:24.129988 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:24.139538 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:24.148530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 628/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:24.154216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.263096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480265b6-6fd6-48b6-b79e-bdbf187aeb3d None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.298933 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642870ba-708a-4b90-9807-87727e218094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.299718 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642870ba-708a-4b90-9807-87727e218094 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.300254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642870ba-708a-4b90-9807-87727e218094 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.300561 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642870ba-708a-4b90-9807-87727e218094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.312057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642870ba-708a-4b90-9807-87727e218094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:24.313547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642870ba-708a-4b90-9807-87727e218094 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-642870ba-708a-4b90-9807-87727e218094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:24.330389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:24.331417 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:24.332633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:24.332633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:24.332633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:24.332633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:24.332633 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 641/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:24.338692 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.450260 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f42494-997d-4e04-ab06-437b43590d01 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.485619 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.486351 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.486646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.486930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.487680 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:24.487993 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:24.489885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:24.502430 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 629/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:24.508358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642870ba-708a-4b90-9807-87727e218094 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.542059 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:24.613249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642870ba-708a-4b90-9807-87727e218094 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.651219 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.652005 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.652324 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.652642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.660936 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:24.662841 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:24.678000 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:24.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:24.680142 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 642/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:24.685165 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.745632 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:24.790010 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b2762-2a51-40ce-9e90-3ceca06b2aee None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.825159 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.825821 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.826109 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.826390 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.827178 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:24.827493 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:24.828960 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f600e4-af86-476b-b7eb-81fc14250942 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:24.841552 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 630/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:24.847547 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:24.942096 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:24.952504 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b620f-d570-49d4-aaf8-045b734cfcb3 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:24.996220 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:24.997056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] REQUEST_METHOD: `PUT` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:24.997348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:24.997629 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] PATH_INFO: `/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/d1990383854d4f97b2d2f4bdd53a05f8/roles/771c065274f04f49bcd4641d79a0e490` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:24.999609 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] RBAC: Authorizing `identity:create_grant(project_id=746ecf0f15fb4b4fa53269fa15460168, user_id=d1990383854d4f97b2d2f4bdd53a05f8, role_id=771c065274f04f49bcd4641d79a0e490)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:25.001102 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e0639-2bf2-45e2-bbdd-76244e001644 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:25.015549 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 643/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:00:24 2022] PUT /identity/v3/projects/746ecf0f15fb4b4fa53269fa15460168/users/d1990383854d4f97b2d2f4bdd53a05f8/roles/771c065274f04f49bcd4641d79a0e490 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:00:27.299541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c6e-914a-4735-9eff-1be2130646d3 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:27.300370 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c6e-914a-4735-9eff-1be2130646d3 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:27.300500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043c6e-914a-4735-9eff-1be2130646d3 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:27.302291 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 631/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:27.309064 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:27.309613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:27.309613 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:27.326736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:27.327938 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:27.397273 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 644/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:27.421894 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043c6e-914a-4735-9eff-1be2130646d3 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:27.527655 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043c6e-914a-4735-9eff-1be2130646d3 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:27.545460 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:27.568445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:27.569273 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:27.569560 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:27.569839 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:27.581015 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:00:27.589180 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:27.590228 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:27.591038 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 632/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:27.596642 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:27.685910 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:27.705559 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a0f0-0893-444a-8c65-bcdee1aa5185 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:27.757435 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:27.758115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:27.758405 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:27.758686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:27.759440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:27.759749 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:27.761185 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:27.769477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 645/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:27.775389 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:27.882577 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83082e-ff15-409c-9976-d43e6a68365a None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:27.916738 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:27.917541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:27.917829 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:27.918108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:27.929829 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:27.931362 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:27.955037 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:27.956145 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:27.957318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:27.957318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:27.957318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:27.957318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:27.957318 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 633/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:27.962632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:28.078562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab81356-5548-4758-8289-fc9a7c051cc0 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:28.120458 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:28.121160 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:28.121446 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:28.121731 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] PATH_INFO: `/v3/domains` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:28.122465 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:28.122769 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:28.124233 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:28.142421 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 646/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:28.149093 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:28.181274 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:28.257871 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fb0e3-91fa-4e30-9ade-186125264791 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:28.292641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:28.293434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:28.293725 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:28.294007 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:28.303815 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:28.305383 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:00:28.321108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:28.322134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:28.328111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 634/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:28.328843 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:28.332668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:28.437335 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fe89e8-02e6-43d0-a656-39817d6828fd None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:28.473667 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:28.474344 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:28.474631 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:28.474908 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] PATH_INFO: `/v3/users` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:28.475747 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:28.476063 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:28.477536 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:28.495347 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 647/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:00:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:28.501203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:28.594851 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:28.605533 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc7566-130b-4dfa-b072-c7a153ec2b85 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:28.640541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:28.641192 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:28.641475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:28.641751 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:28.711681 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:28.713088 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:28.804697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:28.815195 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 635/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:00:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:00:28.821655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:28.883359 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:28.922870 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd19d18-83a6-473d-bf71-398eb62ac0e4 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:28.958947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:28.959740 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:28.960026 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:28.960315 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:28.970423 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:28.971811 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:00:28.990954 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:28.992047 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:28.993141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:00:28.993141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:00:28.993141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:00:28.993141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:28.993141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 648/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:28.998076 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:29.110110 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314e7c-d728-47fd-8d75-26f11c5f6802 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:29.144264 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:29.145481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:29.145880 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:29.146167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] PATH_INFO: `/v3/domains` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:29.146904 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:29.147222 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:29.148640 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:29.161100 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 636/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:29.166738 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:29.269217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e988bf0-de8e-4f39-9fc6-7f367ee6d47e None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:29.283524 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:29.305719 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:29.306529 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:29.306819 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:29.307098 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] PATH_INFO: `/v3/projects/service` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:29.314061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:29.315522 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:00:29.329842 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application with Lock( May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:29.332953 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: ERROR keystone.server.flask.application May 19 20:00:29.334531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 649/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:29.337782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:29.394276 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:29.447113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e73ed-6d9e-464f-90bd-e8acbc4e3e5d None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:29.482012 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:29.482673 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:29.482962 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:29.483240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] PATH_INFO: `/v3/projects` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:29.484029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:29.484344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:29.485823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d12fd3a-f331-4bd2-ac4a-d8869055e5ac None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:29.497669 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 637/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:29.504516 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:29.610146 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eab8b-7f14-4be9-b743-08a3a3e1743b None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:29.645041 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:29.645641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:29.645922 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:29.646194 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:29.647015 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] RBAC: Adding query filter params (role.id=771c065274f04f49bcd4641d79a0e490, scope.project.id=746ecf0f15fb4b4fa53269fa15460168, user.id=d1990383854d4f97b2d2f4bdd53a05f8) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:29.647323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:29.648790 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707c69b-a7da-4d61-abd6-b5c133d08c1d None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:29.654981 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 650/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:00:29 2022] GET /identity/v3/role_assignments?user.id=d1990383854d4f97b2d2f4bdd53a05f8&scope.project.id=746ecf0f15fb4b4fa53269fa15460168&role.id=771c065274f04f49bcd4641d79a0e490 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:31.933982 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1952ab-1a82-4753-867b-cf1d6c733fc0 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:31.934500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1952ab-1a82-4753-867b-cf1d6c733fc0 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:31.934586 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1952ab-1a82-4753-867b-cf1d6c733fc0 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:31.935931 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 638/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:31.941591 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:31.942155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:31.942337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:31.960229 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:31.961464 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:31.991141 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:32.031621 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 651/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:32.064934 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1952ab-1a82-4753-867b-cf1d6c733fc0 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:32.183780 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1952ab-1a82-4753-867b-cf1d6c733fc0 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:32.194379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:32.219400 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:32.220193 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:32.220506 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:32.220786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:32.221467 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:32.223469 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:32.235399 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:32.236353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:32.237324 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 639/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:00:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:32.242836 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:32.354196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44418c89-e812-4f84-a537-2e91fae46abc None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:32.414323 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:32.415057 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:32.415348 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:32.415632 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:32.416443 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:32.416893 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:32.418379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898216d-35f4-44fd-a5bd-d06b3a6bf158 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:32.424081 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 652/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:00:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:32.430111 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:32.550591 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22cd28-8ea3-4907-ba45-14fd23fcab27 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:32.593974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:32.594638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:32.594926 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:32.595203 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] PATH_INFO: `/v3/services` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:32.595914 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] RBAC: Adding query filter params (type=placement) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:32.596225 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:32.598154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea417c44-df76-4e73-9be2-24e889297c61 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:32.603179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 640/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:00:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:34.851105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e8d5-92ef-4d2c-89d6-64f016873ffa None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:34.851618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e8d5-92ef-4d2c-89d6-64f016873ffa None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:34.851618 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e8d5-92ef-4d2c-89d6-64f016873ffa None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:34.853001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 653/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:34.858246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3c03-79b8-49c1-94cd-45f3437f9360 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:34.858758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3c03-79b8-49c1-94cd-45f3437f9360 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:34.858849 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb3c03-79b8-49c1-94cd-45f3437f9360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:34.876794 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-b0eb3c03-79b8-49c1-94cd-45f3437f9360 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:34.878020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-b0eb3c03-79b8-49c1-94cd-45f3437f9360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:34.908103 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:34.948150 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 641/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:34.975238 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e8d5-92ef-4d2c-89d6-64f016873ffa None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:35.007130 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:35.092897 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e8d5-92ef-4d2c-89d6-64f016873ffa None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:35.131438 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:35.131992 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:35.132276 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:35.132593 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:35.133336 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:35.134720 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b044e-4f0b-4bb4-884e-454f35a60031 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:35.147638 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 654/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:00:34 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:00:37.424761 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760c89-706c-44cd-823a-a89a6be5affa None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:37.425554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760c89-706c-44cd-823a-a89a6be5affa None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:37.425733 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99760c89-706c-44cd-823a-a89a6be5affa None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:37.427079 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 642/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:37.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:37.432887 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:37.433033 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:37.450358 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:37.451550 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:37.527500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 655/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:37.551621 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99760c89-706c-44cd-823a-a89a6be5affa None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:37.673823 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99760c89-706c-44cd-823a-a89a6be5affa None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:37.710329 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:37.711042 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:37.711322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:37.711594 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:37.712246 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:37.713638 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:37.725989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:37.727221 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:37.728280 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 643/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:00:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:37.733563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:37.752637 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:37.842292 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8754c-35c3-43b2-9fec-d7154877d6ec None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:37.875983 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:37.876677 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:37.876961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:37.877243 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:37.877961 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:37.878271 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:37.880389 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:37.886686 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 656/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:00:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:37.892706 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:37.924890 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:38.007483 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538a497-1c54-4749-8b3f-689ef1ce20c4 None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:38.042316 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:38.042989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:38.043272 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:38.043545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:38.044344 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] RBAC: Adding query filter params (interface=public, service_id=61ac70698e774058a8a5886e5523eb58, region_id=RegionOne) {{(pid=150738) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:38.044689 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:38.046070 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137be56-f6fa-49fb-87a5-b47876bb50ab None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:38.052348 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 644/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:00:37 2022] GET /identity/v3/endpoints?service_id=61ac70698e774058a8a5886e5523eb58&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:38.058069 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:38.142570 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:38.162105 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f957a-55c7-4607-8e8c-92d662a98d75 None admin] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:38.198692 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:38.199366 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:38.199653 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:38.199939 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:38.200673 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:38.202055 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e52fb4-e22f-410f-9b6c-04552248c557 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:38.209100 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 657/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:00:38 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:00:40.458453 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1898-356c-4747-ad43-38986b8e513a None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:40.459030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1898-356c-4747-ad43-38986b8e513a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:40.459030 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac1898-356c-4747-ad43-38986b8e513a None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:40.460631 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 645/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:00:40.465827 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:40.466265 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:40.466395 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:40.492301 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:00:40.493557 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:00:40.567269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 658/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:00:40 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:00:40.591296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac1898-356c-4747-ad43-38986b8e513a None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:40.692155 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:40.700554 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac1898-356c-4747-ad43-38986b8e513a None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:40.734341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f42b05-cace-4021-a856-1242010287ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:40.735096 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42b05-cace-4021-a856-1242010287ff None admin] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:40.735373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42b05-cace-4021-a856-1242010287ff None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:40.735646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f42b05-cace-4021-a856-1242010287ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:40.736290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f42b05-cace-4021-a856-1242010287ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:40.737788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f42b05-cace-4021-a856-1242010287ff None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application [None req-87f42b05-cace-4021-a856-1242010287ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:00:40.751292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:00:40.752611 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application with Lock( May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self._enter() May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application return self.creator() May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application created_value = creator( May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: ERROR keystone.server.flask.application May 19 20:00:40.753716 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 646/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:00:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:00:40.759545 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:40.837815 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:40.862154 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc642ed-eb73-434e-bff7-51e70030fa1c None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:40.899003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:40.900193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:40.900743 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:40.901159 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] PATH_INFO: `/v3/services` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:40.902274 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] RBAC: Adding query filter params (name=placement) {{(pid=150737) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:00:40.902733 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:40.904775 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9295321-b693-4fec-833b-f17211567d04 None admin] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:40.911418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 659/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:00:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:00:40.917279 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f42b05-cace-4021-a856-1242010287ff None admin] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:00:41.019043 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f42b05-cace-4021-a856-1242010287ff None admin] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:00:41.053747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af21adf4b88843c7988a1c2761d5763b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:00:41.054414 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:00:41.054700 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:00:41.054972 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:00:41.055750 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:00:41.057186 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bd4e3-814a-4688-8beb-b765a384e2fe None admin] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:00:41.075974 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 647/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:00:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:00:50.907272 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=30 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:00:51.071631 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:48.770440 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad970a-8062-4992-900d-de9b4504690e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:48.771003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad970a-8062-4992-900d-de9b4504690e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:48.771003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad970a-8062-4992-900d-de9b4504690e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:48.778277 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9d5c-dcee-443c-a3a2-40858f63e8f7 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:48.778722 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9d5c-dcee-443c-a3a2-40858f63e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:48.778984 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf9d5c-dcee-443c-a3a2-40858f63e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:48.782562 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 660/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:01:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:01:48.954767 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-05bf9d5c-dcee-443c-a3a2-40858f63e8f7 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:01:48.956029 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-05bf9d5c-dcee-443c-a3a2-40858f63e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:49.201950 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 648/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:01:49.278656 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:49.279120 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:49.279515 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:49.281278 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 661/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 20:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:01:49.288166 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905e1b-d020-4ae0-8d57-c6e45df052bf None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:49.288644 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905e1b-d020-4ae0-8d57-c6e45df052bf None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:49.289042 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da905e1b-d020-4ae0-8d57-c6e45df052bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:49.438392 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-da905e1b-d020-4ae0-8d57-c6e45df052bf None None] MFA Rules not processed for user `82dd081413a44b77b8d1342f2755ce51`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:01:49.439757 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-da905e1b-d020-4ae0-8d57-c6e45df052bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:49.476727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:49.566445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 649/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 20:01:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:01:49.576644 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:01:49.578581 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:49.836062 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284e5dc-bf12-47cb-aed6-0e1970f252fb None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:01:49.987579 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:01:49.989002 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:49.989285 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:49.989722 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:49.994736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:50.114521 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:50.190205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:01:50.191858 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a12587-2fa6-41d7-96a6-a40316d88bf2 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:01:50.213943 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 662/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:01:52.983501 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:52.984020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:52.984020 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:52.985646 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 650/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:01:52.992658 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03434632-0336-4009-bfda-c5db717bbfbc None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:52.993655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03434632-0336-4009-bfda-c5db717bbfbc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:52.993925 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03434632-0336-4009-bfda-c5db717bbfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:53.104668 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-03434632-0336-4009-bfda-c5db717bbfbc None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:01:53.105847 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-03434632-0336-4009-bfda-c5db717bbfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:53.230520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 663/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:01:53.269114 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:01:53.270829 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:53.392129 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677b6b-044a-49a9-87a3-2677ba4ced2f None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:01:53.478697 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:53.481544 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:01:53.482292 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:53.482951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:53.483475 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:53.485753 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:53.630802 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:01:53.632707 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799a901-6815-4f25-bafb-9272ddb13a62 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:01:53.657177 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 651/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:01:56.369772 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:56.370288 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:56.370379 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:56.371715 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 664/1315] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:01:56.377056 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b48b29-eeb8-4bf4-985c-31b6ce9250b9 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:56.377481 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b48b29-eeb8-4bf4-985c-31b6ce9250b9 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:56.377628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b48b29-eeb8-4bf4-985c-31b6ce9250b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:56.456341 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-58b48b29-eeb8-4bf4-985c-31b6ce9250b9 None None] MFA Rules not processed for user `4e2602f304894d35a33730dfcd6cc0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:01:56.457902 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-58b48b29-eeb8-4bf4-985c-31b6ce9250b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:56.514739 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:56.567117 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 652/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:01:56 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:01:56.612881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:01:56.615038 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:56.625161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:56.731603 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224b6c1-d20b-416c-ac0e-d7697e2a487e None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:01:56.821356 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:01:56.822045 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:56.822311 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:56.822592 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:56.824372 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:01:56.892527 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:01:56.944099 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:01:56.945604 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264e6a9-e9f2-4e04-b8b1-da019de033c0 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:01:56.968115 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 665/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:01:56 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:01:59.870686 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:59.871190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:59.871190 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:59.872602 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 653/1318] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:01:59.877797 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa92cc-354b-4af1-8214-388e6727564d None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:01:59.878249 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa92cc-354b-4af1-8214-388e6727564d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:01:59.878345 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa92cc-354b-4af1-8214-388e6727564d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:01:59.946650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-95fa92cc-354b-4af1-8214-388e6727564d None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:01:59.947863 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-95fa92cc-354b-4af1-8214-388e6727564d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:00.057102 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 666/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:00.097265 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:00.099541 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:00.220803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf417587-d830-4a68-8046-7c00ed49f79e None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:00.299803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:00.300529 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:00.300876 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:00.301419 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:00.303240 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:00.311874 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:00.420227 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:00.421727 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca359f-a8b7-44cb-8a58-cb7fc323b482 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:00.440269 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 654/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:03.415040 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:03.415563 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:03.415640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:03.417074 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 667/1321] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:03.422899 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05945-cd68-45db-8ff1-a00e6298c730 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:03.423465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05945-cd68-45db-8ff1-a00e6298c730 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:03.423465 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05945-cd68-45db-8ff1-a00e6298c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:03.494354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-c3d05945-cd68-45db-8ff1-a00e6298c730 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:03.495593 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-c3d05945-cd68-45db-8ff1-a00e6298c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:03.560977 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:03.605346 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 655/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:02:03.650753 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:03.652392 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:03.675479 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:03.781205 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae92d4-463e-45c6-8375-c3dfadcf3edc None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:03.874306 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:03.874965 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:03.875259 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:03.875538 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:03.877254 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:03.970773 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:04.016736 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:04.018196 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621af35a-0a4f-4ff6-8b65-2e967ef01500 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:04.038920 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 668/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:06.525413 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:06.525900 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:06.525995 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:06.527390 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 656/1324] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:06.532865 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec9db-5dd8-47f2-8ca2-83d2044d71e8 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:06.533283 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec9db-5dd8-47f2-8ca2-83d2044d71e8 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:06.533432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ec9db-5dd8-47f2-8ca2-83d2044d71e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:06.618564 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-8e7ec9db-5dd8-47f2-8ca2-83d2044d71e8 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:06.619791 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-8e7ec9db-5dd8-47f2-8ca2-83d2044d71e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:06.726109 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 669/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:06.771626 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:06.773296 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:06.891813 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f560eb3a-bb4b-4812-8ea2-2adfe206e1e6 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:06.978390 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:06.979058 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:06.979347 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:06.979758 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:06.981398 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:06.998575 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:07.101197 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:07.102570 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c4a91-0875-4f12-b77f-659055221935 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:07.122809 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 657/1326] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:09.509700 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:09.510222 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:09.510310 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:09.511693 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 670/1327] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:09.518167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d273a5-a531-4511-b045-6d82e4318761 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:09.518167 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d273a5-a531-4511-b045-6d82e4318761 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:09.518606 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d273a5-a531-4511-b045-6d82e4318761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:09.588263 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-61d273a5-a531-4511-b045-6d82e4318761 None None] MFA Rules not processed for user `4e2602f304894d35a33730dfcd6cc0cc`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:09.589530 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-61d273a5-a531-4511-b045-6d82e4318761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:09.658558 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:09.694199 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 658/1328] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:09.731930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:09.733512 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:09.765431 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:09.846217 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad081a3-cb5c-4ac9-a324-297f6f57a6ce None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:09.925508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:09.925508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:09.925508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:09.925508 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:09.925948 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:10.005835 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:10.043689 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:10.045195 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53923b68-d8b8-4f26-86e0-bd41e7330ccf service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:10.064361 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 671/1329] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:12.603830 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:12.604426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:12.604426 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:12.605841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 659/1330] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:12.611329 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b67e-af24-4370-b5ae-11a48aadc0e5 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:12.611782 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b67e-af24-4370-b5ae-11a48aadc0e5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:12.611881 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320b67e-af24-4370-b5ae-11a48aadc0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:12.680572 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-d320b67e-af24-4370-b5ae-11a48aadc0e5 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:12.681793 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-d320b67e-af24-4370-b5ae-11a48aadc0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:12.783558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 672/1331] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:12.819922 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:12.821524 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:12.928831 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6522d-33fd-471c-ab81-b8b393faf616 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:13.003353 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:13.004592 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:13.004911 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:13.005187 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:13.006930 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:13.034490 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:13.123833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:13.125317 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c5640-7deb-4bfb-8356-0df7be08da83 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:13.143989 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 660/1332] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:15.464772 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:15.465310 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:15.465408 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:15.466735 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 673/1333] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:15.472063 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567d3b-d65a-448a-ac86-40e9b73d74ea None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:15.472466 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567d3b-d65a-448a-ac86-40e9b73d74ea None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:15.472693 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06567d3b-d65a-448a-ac86-40e9b73d74ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:15.546551 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-06567d3b-d65a-448a-ac86-40e9b73d74ea None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:15.547788 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-06567d3b-d65a-448a-ac86-40e9b73d74ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:15.627841 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:15.652232 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 661/1334] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:15.694086 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:15.695626 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:15.738682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:15.806493 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec44481-f114-44c1-a2fc-4d06ff453688 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:15.884589 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:15.885641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:15.885641 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:15.885856 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:15.887398 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:15.971825 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:16.000327 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:16.001801 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a9e32-171b-4360-b850-e16281460d63 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:16.020262 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 674/1335] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:18.649134 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:18.649683 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:18.649683 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:18.651094 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 662/1336] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:18.656590 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72440-a9ef-461c-9dd8-ae151926fcd0 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:18.657058 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72440-a9ef-461c-9dd8-ae151926fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:18.657158 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72440-a9ef-461c-9dd8-ae151926fcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:18.726539 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-02b72440-a9ef-461c-9dd8-ae151926fcd0 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:18.727774 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-02b72440-a9ef-461c-9dd8-ae151926fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:18.828293 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 675/1337] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:18.866067 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:18.869290 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:18.986115 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3a1ed-13f6-4b1b-8333-a2bdffc6a8ef None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:19.056517 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:19.057411 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:19.057786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:19.058445 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:19.060628 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:19.130532 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=100 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:19.181921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:19.183368 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094fd87-3731-42fd-858f-13730dbd9255 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:19.204146 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 663/1338] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:23.575019 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:23.575574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:23.575574 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:23.577013 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 676/1339] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:23.583212 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4e42-8e67-4944-8da1-4238c0ab623f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:23.583664 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4e42-8e67-4944-8da1-4238c0ab623f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:23.583854 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4e42-8e67-4944-8da1-4238c0ab623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:23.634883 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-dc5b4e42-8e67-4944-8da1-4238c0ab623f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:23.636019 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-dc5b4e42-8e67-4944-8da1-4238c0ab623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:23.717968 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 664/1340] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:23.758135 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:23.760307 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:23.784056 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba857c3c-d4b5-472e-9118-e1b71a5d887b None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:23.788174 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:23.788877 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:23.789161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:23.789432 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:23.799380 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:23.801903 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76161d-459b-4527-adbd-3d1ee6d1e1f4 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:23.811507 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 677/1341] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:26.791918 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8ffe-30f2-4923-9a6c-9787a219b9ac None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:26.792486 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8ffe-30f2-4923-9a6c-9787a219b9ac None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:26.792590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d8ffe-30f2-4923-9a6c-9787a219b9ac None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:26.793955 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 665/1342] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:26.803541 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e238e7-bf60-4804-9ffb-1a3d2770c5a2 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:26.804106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e238e7-bf60-4804-9ffb-1a3d2770c5a2 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:26.804106 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e238e7-bf60-4804-9ffb-1a3d2770c5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:26.834182 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-24e238e7-bf60-4804-9ffb-1a3d2770c5a2 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:26.835500 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-24e238e7-bf60-4804-9ffb-1a3d2770c5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:26.843560 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 678/1343] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:26.881576 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d8ffe-30f2-4923-9a6c-9787a219b9ac None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:26.892698 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d8ffe-30f2-4923-9a6c-9787a219b9ac None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:26.896991 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:26.897641 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:26.897932 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:26.898213 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:26.908565 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:26.911434 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c8076-8d70-4159-9800-e1401f4790c7 service neutron] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:26.921951 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 666/1344] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:31.304979 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a207b1-86bf-410f-9825-0dc3e35b38ec None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:31.305578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a207b1-86bf-410f-9825-0dc3e35b38ec None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:31.305578 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a207b1-86bf-410f-9825-0dc3e35b38ec None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:31.306941 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 679/1345] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:31.312121 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b43a5-c8f8-45a1-989b-8d83c026376b None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:31.312588 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b43a5-c8f8-45a1-989b-8d83c026376b None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:31.312711 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472b43a5-c8f8-45a1-989b-8d83c026376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:31.331154 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-472b43a5-c8f8-45a1-989b-8d83c026376b None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:31.334452 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-472b43a5-c8f8-45a1-989b-8d83c026376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:31.343026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 667/1346] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:02:31.388206 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a207b1-86bf-410f-9825-0dc3e35b38ec None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:31.401331 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a207b1-86bf-410f-9825-0dc3e35b38ec None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:31.404930 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82dd081413a44b77b8d1342f2755ce51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:31.405586 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:31.405871 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:31.406155 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:31.412795 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:31.417727 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:31.419172 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f14b2-041c-4b4b-b277-2d90e1b3bef7 service neutron] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:31.429682 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 680/1347] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:37.114833 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5e11-fea5-45ee-b5a9-6e15870506fe None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:37.115325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5e11-fea5-45ee-b5a9-6e15870506fe None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:37.115428 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5e11-fea5-45ee-b5a9-6e15870506fe None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:37.116940 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 668/1348] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:37.122161 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749b66-f2a5-41cd-9585-3051e300f10f None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:37.122978 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749b66-f2a5-41cd-9585-3051e300f10f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:37.123442 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a749b66-f2a5-41cd-9585-3051e300f10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:37.294724 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-4a749b66-f2a5-41cd-9585-3051e300f10f None None] MFA Rules not processed for user `d1990383854d4f97b2d2f4bdd53a05f8`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:37.296001 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-4a749b66-f2a5-41cd-9585-3051e300f10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:37.392194 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 681/1349] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:02:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:41.325909 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=80 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:41.994223 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c7a2-21a0-40f2-b5e2-034333c3340e None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:41.994518 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c7a2-21a0-40f2-b5e2-034333c3340e None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:41.994803 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c7a2-21a0-40f2-b5e2-034333c3340e None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:41.996373 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 669/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:02:42.001640 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2675f-9811-48d5-b63c-8d2597e04dc7 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:42.002519 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2675f-9811-48d5-b63c-8d2597e04dc7 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:42.002882 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2675f-9811-48d5-b63c-8d2597e04dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:42.022520 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-2ba2675f-9811-48d5-b63c-8d2597e04dc7 None None] MFA Rules not processed for user `d1990383854d4f97b2d2f4bdd53a05f8`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:42.023914 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-2ba2675f-9811-48d5-b63c-8d2597e04dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:42.033368 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 682/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:02:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:43.639281 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572d20-e0b1-41a7-a28a-44678ab69f14 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:43.640008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572d20-e0b1-41a7-a28a-44678ab69f14 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:43.640008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61572d20-e0b1-41a7-a28a-44678ab69f14 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:43.642064 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 670/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:43.648309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe7fd-79ba-4c29-afbd-80e47d801936 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:43.648915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe7fd-79ba-4c29-afbd-80e47d801936 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:43.648915 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe7fd-79ba-4c29-afbd-80e47d801936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:43.667820 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-bcdfe7fd-79ba-4c29-afbd-80e47d801936 None None] MFA Rules not processed for user `d1990383854d4f97b2d2f4bdd53a05f8`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:43.669061 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-bcdfe7fd-79ba-4c29-afbd-80e47d801936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:43.679763 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 683/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:47.632297 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da68fd-afb1-4bab-ada3-12088fdd6b33 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:47.632814 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da68fd-afb1-4bab-ada3-12088fdd6b33 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:47.632892 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da68fd-afb1-4bab-ada3-12088fdd6b33 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:47.635590 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 671/1354] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:47.643179 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df98f1-921e-4dc5-bb0b-709b70f65571 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:47.643800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df98f1-921e-4dc5-bb0b-709b70f65571 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:47.643800 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df98f1-921e-4dc5-bb0b-709b70f65571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:47.677601 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-f9df98f1-921e-4dc5-bb0b-709b70f65571 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:47.679393 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-f9df98f1-921e-4dc5-bb0b-709b70f65571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:47.688655 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 684/1355] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:47.747724 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e798050-5269-4928-a2e3-441c448aefb8 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:47.748289 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e798050-5269-4928-a2e3-441c448aefb8 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:47.748782 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e798050-5269-4928-a2e3-441c448aefb8 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:47.750379 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 672/1356] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:47.767269 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9294-3cb2-4465-bb94-15d3677a27e5 None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:47.767989 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9294-3cb2-4465-bb94-15d3677a27e5 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:47.769312 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a9294-3cb2-4465-bb94-15d3677a27e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:47.832585 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-3d8a9294-3cb2-4465-bb94-15d3677a27e5 None None] MFA Rules not processed for user `482a2286f928483b8bce71f07fb4809f`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:47.833812 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-3d8a9294-3cb2-4465-bb94-15d3677a27e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:47.889283 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=100 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:02:47.932528 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 685/1357] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:47.940572 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e798050-5269-4928-a2e3-441c448aefb8 None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:47.956322 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e798050-5269-4928-a2e3-441c448aefb8 None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:47.962130 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482a2286f928483b8bce71f07fb4809f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:47.962834 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:47.963125 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:47.963581 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:47.973152 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:47.974786 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2f792-dab5-4af1-b5b5-270f9bcb2f74 service nova] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:47.984885 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 673/1358] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:50.532946 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fda63-9637-429e-82ea-f8ce559deeae None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:50.533486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fda63-9637-429e-82ea-f8ce559deeae None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:50.533486 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fda63-9637-429e-82ea-f8ce559deeae None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:50.535761 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 686/1359] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:50.541970 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a4-ebe6-44e9-8b78-1bf8761a9126 None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:50.542418 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a4-ebe6-44e9-8b78-1bf8761a9126 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:50.542545 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad111a4-ebe6-44e9-8b78-1bf8761a9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:50.562108 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2ad111a4-ebe6-44e9-8b78-1bf8761a9126 None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:50.563325 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2ad111a4-ebe6-44e9-8b78-1bf8761a9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:50.573354 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 674/1360] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:50.618739 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fda63-9637-429e-82ea-f8ce559deeae None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:50.641449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fda63-9637-429e-82ea-f8ce559deeae None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:50.648947 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482a2286f928483b8bce71f07fb4809f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:50.649244 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:50.649531 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:50.649899 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:50.660994 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:50.662659 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c661167-6110-4a91-b46c-1d6c370a530e service nova] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:50.673984 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 687/1361] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:02:53.339576 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cb30a-4167-4e9c-b4a3-1dea75720ab4 None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:53.340150 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cb30a-4167-4e9c-b4a3-1dea75720ab4 None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:53.340150 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57cb30a-4167-4e9c-b4a3-1dea75720ab4 None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:53.341595 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 675/1362] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:53.347803 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee268-a183-4fb8-954e-53d1c3cba93f None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:53.348216 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee268-a183-4fb8-954e-53d1c3cba93f None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:53.348434 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee268-a183-4fb8-954e-53d1c3cba93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:53.368652 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-c98ee268-a183-4fb8-954e-53d1c3cba93f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:53.369904 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-c98ee268-a183-4fb8-954e-53d1c3cba93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:53.378449 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 688/1363] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:53.425597 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b0ae-519a-4360-a4f7-006eaf4cb95d None None] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:53.426314 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b0ae-519a-4360-a4f7-006eaf4cb95d None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:53.426931 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024b0ae-519a-4360-a4f7-006eaf4cb95d None None] PATH_INFO: `/` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:53.428747 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 676/1364] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:02:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:53.434635 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c6e7e-d756-4769-89d9-83256d3b429d None None] REQUEST_METHOD: `POST` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:53.435224 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c6e7e-d756-4769-89d9-83256d3b429d None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:53.435511 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c6e7e-d756-4769-89d9-83256d3b429d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:53.455406 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.auth.core [None req-087c6e7e-d756-4769-89d9-83256d3b429d None None] MFA Rules not processed for user `482a2286f928483b8bce71f07fb4809f`. Rule list: `[]` (Enabled: `True`). {{(pid=150737) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:53.456789 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.fernet_utils [None req-087c6e7e-d756-4769-89d9-83256d3b429d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150737) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:53.472608 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 689/1365] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:02:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:53.476289 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024b0ae-519a-4360-a4f7-006eaf4cb95d None None] Authenticating user token {{(pid=150738) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:53.494956 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024b0ae-519a-4360-a4f7-006eaf4cb95d None None] Validating token access rules against request {{(pid=150738) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:53.501783 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482a2286f928483b8bce71f07fb4809f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150738) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:53.502278 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] REQUEST_METHOD: `GET` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:53.502568 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:53.502858 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:53.516557 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:53.518179 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb1256-b4d8-4a59-9acb-8659920968d3 service nova] RBAC: Authorization granted {{(pid=150738) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:53.530122 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 677/1366] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:02:54.489573 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47bdf6-fac4-4e63-909b-f50832ab4a06 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:54.490131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47bdf6-fac4-4e63-909b-f50832ab4a06 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:54.490131 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47bdf6-fac4-4e63-909b-f50832ab4a06 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:54.491852 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 690/1367] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:54.497502 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267bf0-e6af-49a6-aaf4-2608ba776c2a None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:54.498018 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267bf0-e6af-49a6-aaf4-2608ba776c2a None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:54.498113 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf267bf0-e6af-49a6-aaf4-2608ba776c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:54.522378 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-cf267bf0-e6af-49a6-aaf4-2608ba776c2a None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:54.523612 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-cf267bf0-e6af-49a6-aaf4-2608ba776c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:54.531921 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 678/1368] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:56.029433 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8b31-6474-448f-80f0-6325678ea755 None None] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:56.030003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8b31-6474-448f-80f0-6325678ea755 None None] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:56.030003 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e8b31-6474-448f-80f0-6325678ea755 None None] PATH_INFO: `/` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:56.031418 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 691/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:02:56.038008 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275445a-ee91-4037-b49b-aeb52b8ccb3f None None] REQUEST_METHOD: `POST` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:56.038500 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275445a-ee91-4037-b49b-aeb52b8ccb3f None None] SCRIPT_NAME: `/identity` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:56.038610 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275445a-ee91-4037-b49b-aeb52b8ccb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=150738) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:56.057349 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.auth.core [None req-2275445a-ee91-4037-b49b-aeb52b8ccb3f None None] MFA Rules not processed for user `af21adf4b88843c7988a1c2761d5763b`. Rule list: `[]` (Enabled: `True`). {{(pid=150738) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:02:56.058616 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG keystone.common.fernet_utils [None req-2275445a-ee91-4037-b49b-aeb52b8ccb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=150738) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:02:56.067665 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: [pid: 150738|app: 0|req: 679/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:02:56.117124 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e8b31-6474-448f-80f0-6325678ea755 None None] Authenticating user token {{(pid=150737) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:02:56.132102 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e8b31-6474-448f-80f0-6325678ea755 None None] Validating token access rules against request {{(pid=150737) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:02:56.137558 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '482a2286f928483b8bce71f07fb4809f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '746ecf0f15fb4b4fa53269fa15460168', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=150737) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:02:56.138193 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] REQUEST_METHOD: `GET` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:02:56.138477 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] SCRIPT_NAME: `/identity` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:02:56.138758 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=150737) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:02:56.148664 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:02:56.150309 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245a83e-9ec8-4748-930d-b80003adef30 service nova] RBAC: Authorization granted {{(pid=150737) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:02:56.161201 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: [pid: 150737|app: 0|req: 692/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:03:06.052026 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: DEBUG dbcounter [-] [150738] Writing DB stats keystone:SELECT=21 {{(pid=150738) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:06.156337 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: DEBUG dbcounter [-] [150737] Writing DB stats keystone:SELECT=31 {{(pid=150737) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:03:18.793154 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: Thu May 19 20:03:18 2022 - graceful shutdown triggered... May 19 20:03:18.793650 dv-cinder-840678-11-iscsi devstack@keystone.service[150737]: Gracefully killing worker 1 (pid: 150737)... May 19 20:03:18.793275 dv-cinder-840678-11-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:03:18.794420 dv-cinder-840678-11-iscsi devstack@keystone.service[150738]: Gracefully killing worker 2 (pid: 150738)... May 19 20:03:19.794389 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: worker 1 buried after 1 seconds May 19 20:03:19.794389 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: worker 2 buried after 1 seconds May 19 20:03:19.794389 dv-cinder-840678-11-iscsi devstack@keystone.service[150726]: goodbye to uWSGI. May 19 20:03:19.795879 dv-cinder-840678-11-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:03:19.796467 dv-cinder-840678-11-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:09:25.696227 dv-cinder-840678-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:09:25.705300 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:09:25 2022] *** May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: nodename: dv-cinder-840678-11-iscsi May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: machine: x86_64 May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: clock source: unix May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: pcre jit disabled May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: detected number of CPU cores: 4 May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: current working directory: / May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: detected binary path: /usr/bin/uwsgi-core May 19 20:09:25.707286 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: your processes number limit is 47680 May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: your memory page size is 4096 bytes May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: detected max file descriptor number: 2048 May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: lock engine: pthread robust mutexes May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: thunder lock: enabled May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:09:25.708132 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:09:25.764525 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: Python main interpreter initialized at 0x561ebc0c69c0 May 19 20:09:25.764525 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: python threads support enabled May 19 20:09:25.764525 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: your server socket listen backlog is limited to 100 connections May 19 20:09:25.764525 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: your mercy for graceful operations on workers is 90 seconds May 19 20:09:25.764955 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:09:25.764955 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: *** Operational MODE: preforking *** May 19 20:09:25.765233 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: *** uWSGI is running in multiple interpreter mode *** May 19 20:09:25.765233 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: spawned uWSGI master process (pid: 214136) May 19 20:09:25.765448 dv-cinder-840678-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:09:25.766271 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: spawned uWSGI worker 1 (pid: 214148, cores: 1) May 19 20:09:25.766377 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: spawned uWSGI worker 2 (pid: 214149, cores: 1) May 19 20:09:25.766535 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:09:31.561288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:09:31.561288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: from cryptography.utils import int_from_bytes May 19 20:09:31.562199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:09:31.562199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: from cryptography.utils import int_from_bytes May 19 20:09:31.610789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:09:31.610789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: from cryptography.utils import int_from_bytes May 19 20:09:31.611569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:09:31.611569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: from cryptography.utils import int_from_bytes May 19 20:09:31.612604 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:09:31.612806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:09:31.613015 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] command line args: [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:09:31.613292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:09:31.613479 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ================================================================================ {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:09:31.613741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] admin_token = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.613996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] config_dir = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.614192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.614384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] config_source = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.614578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] debug = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.614782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.615179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.615354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] insecure_debug = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.615656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.615848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.616089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.616275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_config_append = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.616489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.616701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_dir = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.616959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_file = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.617222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.617417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.617593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.617824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.618008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.618192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.618447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.618701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.618944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.619184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.619426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.619665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.619927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.620186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] notification_format = cadf {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.620506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.621040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.621206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] publish_errors = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.621499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.621744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.621960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.622193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.622451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.622672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] standard_threads = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.622914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] strict_password_check = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.623166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.623351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] use_eventlog = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.623534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] use_journal = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.623715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] use_json = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.623896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] use_stderr = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.624078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] use_syslog = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.624316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] watch_log_file = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.624617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.625191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.625191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.625340 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.625597 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.625845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.626104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.626349 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.626607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.626862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.627121 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.627361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.627621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.627864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.628118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.628373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.628642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.628908 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.629169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.629421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.629665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.629925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.630199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.630458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.630717 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.630955 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.631192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.631429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.631670 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.631926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.632190 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.632464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.632673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.632933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.633188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.633433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.633689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.633926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.634167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.634407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.634643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.634881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.635115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.635351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.635593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.635849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.636087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.636327 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.636596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.636840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.637747 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.638122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.638393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.638582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.638825 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.639073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.639318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.639563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.639802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.640060 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.640303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.640588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.640798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.641069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.641312 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.641552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.641799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.642042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.642322 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.642567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.642823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.643077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.643321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.643568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.643810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.644066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.enabled = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.644311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.644583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.644835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.645079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.645323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.645565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.645809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.646051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.646326 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.646578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.646830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.647078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.647330 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.647576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.647822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.648065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.648569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.648808 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.649051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.649289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.649537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.649781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.650036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.650281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.650678 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.650951 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.651204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.651448 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.external = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.651690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.mapped = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.652000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.652245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.652510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.password = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.653364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] auth.token = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.653608 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.653867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] catalog.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.654085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.654331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.654587 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.654847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.655087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.655332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.655587 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.655846 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.656105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.656351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.656875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.657151 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.657414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.657657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.657921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.658217 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.658470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.658740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.658990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.659254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.659497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.659740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.659992 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.660244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.660504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.660760 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.660916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:09:31.661070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:09:31.661239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.661396 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] command line args: [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:09:31.661544 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.661695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:09:31.661836 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.661993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ================================================================================ {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:09:31.662148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.662298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] admin_token = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.662452 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.662603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] config_dir = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.662750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.662897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.663049 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.663190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] config_source = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.663336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.663483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] debug = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.663483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.663804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.663953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.664108 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.664256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] insecure_debug = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.664452 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.664602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.664758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.664907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.665047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.665186 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.665333 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.665489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_config_append = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.665638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.665795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.665795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_dir = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.666059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.666204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_file = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.666351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.672416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.672590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.672741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.672881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.673028 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.673169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.673329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.673474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.673637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.673782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.673922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.674071 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.674232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.674381 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.674528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.674676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.674825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.674977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.675124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.675295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.675449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.675605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.675747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.675899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.676047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.676193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.676341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] notification_format = cadf {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.676572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.676717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.676869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.676869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.676869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.677280 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.677422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.677422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.677715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] publish_errors = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.677862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.678010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.678298 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.678516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.678516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.678830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.678830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.679190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.679346 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.679506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.679660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.679809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] standard_threads = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.679969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.password = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.680118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] strict_password_check = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.680263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.680436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.680436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] use_eventlog = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.680976 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.681122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] use_journal = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.681267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.681411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] use_json = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.681555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.681707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] use_stderr = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.681707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] use_syslog = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.681981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.682138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] watch_log_file = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:09:31.682288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.682435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.682596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.682743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.682887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683394 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.683916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.684067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.684311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.684311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.684710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.684857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685146 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.685880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.686940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687526 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687665 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687814 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.687959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688631 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.688922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689313 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689752 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.689897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.691933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692837 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.692977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] policy.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693697 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.693851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694164 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] receipt.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.694911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695195 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695762 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.695907 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696684 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.696876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697312 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.697892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] revoke.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698765 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.698904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] role.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.699060 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.699200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] role.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.699337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] role.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700312 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] role.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.700941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701409 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701740 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.701893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.702927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703723 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.703868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704515 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704515 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.704958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705258 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.705868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.enabled = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706633 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706778 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.706928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707396 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707698 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.707999 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.708935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.709845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.710977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.provider = fernet {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.711883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712808 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.712959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] trust.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713580 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.713731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.714927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.external = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.715725 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.715883 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.mapped = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.connection = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.716872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.password = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] auth.token = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.717878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] catalog.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718797 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.718953 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.719868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720165 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.720952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721253 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.721984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722452 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214149) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.722611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.723838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.725133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.726342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.727535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.728795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.password = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.730036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.731392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.732782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.734017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] policy.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.735232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] receipt.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.736484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] revoke.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.737799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] role.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] role.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] role.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] role.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.739020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.740237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.741264 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.741418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.741712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.742954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.provider = fernet {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.744168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] trust.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.745468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747639 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.connection = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.747815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.749772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.750232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.750232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:09:31.750526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=214148) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:09:31.774655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.774872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.776602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.776979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.777117 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.788057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.806922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.807164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.830362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.847612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.882461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.882684 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.893421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.893668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.950454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.950690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.950924 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951149 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951280 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951381 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951826 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.951956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.952181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.974418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.974654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.974882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.974982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975112 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975340 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975663 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.975894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.976021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.976245 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.987743 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:31.988289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.001400 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.001642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.001877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.002107 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.002338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.002581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.002809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214148) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.002991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.003235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.003468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.003699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.003927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.004153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.004376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=214149) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:09:32.021000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.021154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.021242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:09:32.021329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:09:32.021557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.021655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.021757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:09:32.021849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:09:32.022034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022302 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022821 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.022922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.034258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.034552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.034835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.035130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.036054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.036364 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.036684 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.037001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.043552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.045581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.046936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.047226 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.047518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.047936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.048228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.048530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.048810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.048937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.049092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.049228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.049370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.049523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.049651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.049926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.050016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.050221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.050494 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.050772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.051056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.051342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.051626 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.051904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.077761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.079395 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.079688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.080456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.082051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.082627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.085122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.085426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.085717 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.087847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.088145 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.088463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.093751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.094058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.096918 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.097217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.098871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.099166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.099460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.099755 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.100039 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.100320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.100622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.100899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.101903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.102200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.102496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.102789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.103073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.103357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.103636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.103913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.120284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.120602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.123259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.123555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.125588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.128264 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.129248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.129553 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.131946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.132248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.134537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.137996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.138912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.141784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.143895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.144228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.144554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.144851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.145142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.145430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.145736 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.146050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.146349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.146641 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.146925 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.147215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.165206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.165600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.165885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.166177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.167017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.167434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.167710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.167987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.174338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.176074 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.176201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.176358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.177839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.178128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.179455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.179740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.180040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.180441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.180906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.182907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.183200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.183492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.183825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.184126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.195004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.195517 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.196184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.196184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.196184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.196475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.196724 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.198026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.198399 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.198643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.198935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.199223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.200133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.200608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.216147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.219467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.219935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.222815 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.223287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.224446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.226119 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.226417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.226645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.226736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.228270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.229975 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.230266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.230555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.234273 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.235932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.238208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.239259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.239574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.239867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.239956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.240152 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.243300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.243608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.243907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.244197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.249828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.250193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.250486 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.253936 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.254243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.254537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.258435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=214149) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.258729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.259028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.259315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.259592 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.259891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.260170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.260469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.260766 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.261050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.261330 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.261600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.262406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.262849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=214149) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.263284 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=214148) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:09:32.263575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.263862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.264148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.264460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.264745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.265033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.265310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.265596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.265874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.266166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.266443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.266716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.266990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=214148) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:09:32.302749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561ebc0c69c0 pid: 214149 (default app) May 19 20:09:32.306585 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58f1fe-1f2f-4b2c-aec7-a5544e9f3011 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:32.306882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58f1fe-1f2f-4b2c-aec7-a5544e9f3011 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:32.307172 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58f1fe-1f2f-4b2c-aec7-a5544e9f3011 None None] PATH_INFO: `/v3` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:32.307278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x561ebc0c69c0 pid: 214148 (default app) May 19 20:09:32.312163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu May 19 20:09:32 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:40.533495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:40.534035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:40.534035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:40.538326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:40.543849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:40.544291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:40.544447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:40.613463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: INFO dbcounter [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] Registered counter for database keystone May 19 20:09:40.635782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG oslo_db.sqlalchemy.engines [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=214149) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:09:40.643598 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writer thread running {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:09:40.829019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG passlib.handlers.bcrypt [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=214149) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:09:40.829357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG passlib.handlers.bcrypt [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=214149) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:09:40.886562 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:40.889032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:41.020061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 20:09:41.043535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:41.114664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: INFO dbcounter [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] Registered counter for database keystone May 19 20:09:41.135756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG oslo_db.sqlalchemy.engines [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=214148) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:09:41.142607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writer thread running {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:09:41.268110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a012c9-bf37-421c-af9a-37c2aa44ea3d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:41.272550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0064d8382e454dc1bf5476c00fe5a47a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:41.273248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:41.273535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:41.273812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:41.282368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:41.334904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: WARNING keystone.common.rbac_enforcer.enforcer [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:09:41.335248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9915f4-dc1a-4f8b-ba45-304e29ebce0a admin admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:41.345966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 20:09:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 20:09:41.352281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:41.367351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bbfc8-05e8-4a7c-b111-bcdaf4b2daff None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:41.372603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0064d8382e454dc1bf5476c00fe5a47a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:41.373740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:41.374029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:41.374307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] PATH_INFO: `/v3/projects/0064d8382e454dc1bf5476c00fe5a47a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:41.375851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] RBAC: Authorizing `identity:get_project(project_id=0064d8382e454dc1bf5476c00fe5a47a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:41.538855 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:09:41.539196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7e343-f09a-4a72-b283-f9fae5792012 admin admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:41.544939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu May 19 20:09:41 2022] GET /identity/v3/projects/0064d8382e454dc1bf5476c00fe5a47a => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:43.844547 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6940-a724-4f2d-9f05-8791b65681d5 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:43.845094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6940-a724-4f2d-9f05-8791b65681d5 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:43.845094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f6940-a724-4f2d-9f05-8791b65681d5 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:43.849413 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:43.855666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:43.856128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:43.856333 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:43.875889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:43.877457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:43.885372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:09:43.908724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f6940-a724-4f2d-9f05-8791b65681d5 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:43.921193 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f6940-a724-4f2d-9f05-8791b65681d5 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:43.925448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0064d8382e454dc1bf5476c00fe5a47a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:43.926088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:43.926376 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:43.926655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:43.935148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:43.936993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e2ed4-a581-4c68-b5ab-808751e1316e admin admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:43.945756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 20:09:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:09:43.953122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:43.962599 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e576f14-857a-417e-bf77-a82cf954c921 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:43.967347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0064d8382e454dc1bf5476c00fe5a47a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:43.968227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:43.968565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:43.968854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:43.970291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:43.972175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66aa473-5aea-4404-b66f-f488bea5578b admin admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:43.973845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu May 19 20:09:43 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:46.362286 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f071-70d9-4443-ae68-2c52b35b1b3a None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.362841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f071-70d9-4443-ae68-2c52b35b1b3a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.362841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635f071-70d9-4443-ae68-2c52b35b1b3a None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.364214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:46.369676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.370246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.370246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.387687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:46.388923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:46.440131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61540feb-4b8a-4af0-b4cb-d1bdfc42b784 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.440557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61540feb-4b8a-4af0-b4cb-d1bdfc42b784 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.440923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61540feb-4b8a-4af0-b4cb-d1bdfc42b784 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.442372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:09:46.448232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.448486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.448783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.477757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:46.490227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG passlib.handlers.bcrypt [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=214148) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:09:46.490644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG passlib.handlers.bcrypt [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=214148) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:09:46.506059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:46.530914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:46.537972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:46.539678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:46.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8d202-f724-4d1c-88a2-3643b934aded None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:46.644761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:09:46.660034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:46.660743 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.661033 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.661317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.668730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:46.763480 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:46.765122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:46.789947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d82c-3c11-47be-a99b-77e1bf9db2f4 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:46.790711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:46.794227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:46.836841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:46.837917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:46.838209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:46.838493 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:46.850680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:46.863964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:46.865235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:09:46.866146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 8/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:09:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:09:46.872644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:46.879450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 7/15] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:09:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:09:46.885098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:46.979987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563e16e-4c9f-4a2a-a3a0-41bf2c82d321 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:46.992139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27041b-920b-4383-99ca-4475876ee65d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:47.015785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.016324 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:47.016461 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.016688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.016967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.017743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:47.018083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.019572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.029567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.030376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.030662 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.030940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.033711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:09:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:47.039121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:47.041627 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.043470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:09:47.065243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:47.066393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:47.067540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:09:47.067540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:09:47.067540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:09:47.067540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:09:47.067540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:09:47.069928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:47.081574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:47.157944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c6711-2df8-4cea-9a53-ffd76f09a3a0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:47.189513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f5295-8cb9-4b80-a858-5a779a8747da None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:47.198330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.199276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.199567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.199854 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.201725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.203198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.204897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:09:47 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:47.210268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:47.227079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.227774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.228068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.228352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.229156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:47.229469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.231438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.250027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:47.317475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:47.325645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106825cc-6a86-476e-b0e0-fd17beab176e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:47.363231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.364064 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.364357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.364675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.365939 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.367478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f89ecd-91ee-4fd4-a35e-c29b3189ff46 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.369239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:47.375862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:47.414159 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:47.491264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45a73-14c1-4ddb-b2e9-c45cec83b101 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:47.528962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:47.529640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:47.529924 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:47.530204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:47.531063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.domain.id=default, user.id=bfca2eff03f04a12a3f7eb5950753314) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:47.531380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:47.532891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1104e-6702-4181-bfe0-da49b444b67f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:47.540779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 20:09:47 2022] GET /identity/v3/role_assignments?user.id=bfca2eff03f04a12a3f7eb5950753314&scope.domain.id=default&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:49.873073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b460-e99d-46dc-800b-9f0f402573b0 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:49.874006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b460-e99d-46dc-800b-9f0f402573b0 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:49.874605 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4b460-e99d-46dc-800b-9f0f402573b0 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:49.876284 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:49.882511 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7a9c-1dc7-4833-a402-50c4f06f2af4 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:49.883542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7a9c-1dc7-4833-a402-50c4f06f2af4 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:49.884036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e7a9c-1dc7-4833-a402-50c4f06f2af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:49.913674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-3e8e7a9c-1dc7-4833-a402-50c4f06f2af4 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:49.915282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-3e8e7a9c-1dc7-4833-a402-50c4f06f2af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:50.005001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:50.041677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b460-e99d-46dc-800b-9f0f402573b0 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:50.130757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744b9d1-4fda-4540-a537-e24f84ad1e63 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.131405 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744b9d1-4fda-4540-a537-e24f84ad1e63 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.131892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744b9d1-4fda-4540-a537-e24f84ad1e63 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.134022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:50.140364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.141095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.141799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.177547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:50.179303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:50.185306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4b460-e99d-46dc-800b-9f0f402573b0 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:50.250704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:50.270187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:50.271419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.271912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.272391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.307811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:50.309906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:50.411301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:50.458047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:09:50.502931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:50.503946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:50.504923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:09:50.504923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:09:50.504923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:09:50.504923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:09:50.504923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 13/26] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:50.513041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:50.547208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:50.660949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba4f05-e685-4823-ae5a-e59de65bc9d5 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:50.766016 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:50.766645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.766937 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.767229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.768216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:50.769704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc27cffe-3585-497b-b167-b2f9bc254edb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:50.772072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84c417-0418-4d77-9767-0ac998a1e18c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:50.858083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:50.859187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.859713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.860304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.861334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:50.861876 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:50.863703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f88ec-ea08-4a05-9ce7-1eab51d800fd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:50.910722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 14/27] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:50.912096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7446c-5b6d-48f6-8b3c-18a782357112 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.912387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7446c-5b6d-48f6-8b3c-18a782357112 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.912705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7446c-5b6d-48f6-8b3c-18a782357112 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:50.913963 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 15/28] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:50.921619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7446c-5b6d-48f6-8b3c-18a782357112 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:50.960017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 14/29] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:09:50 2022] POST /identity/v3/roles => generated 219 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:09:50.964305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:50.964621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:50.965244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.004158 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:51.006051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:51.033583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:51.156838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:51.158644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 15/30] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:51.180331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7446c-5b6d-48f6-8b3c-18a782357112 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:51.214397 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:51.269590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:51.270281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.270569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.270847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.271590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:51.273115 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:51.368062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e0501-dc58-4426-9902-d8a8571b2fbd None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:51.410775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:51.411462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.411756 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.412039 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.412847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:51.414449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc969ab-bbde-4e44-b375-be3bfab82676 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:51.433129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 16/31] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:09:51 2022] POST /identity/v3/roles => generated 225 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:09:51.435236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982041-31ef-4266-a2d4-311bbdff052b None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.435558 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982041-31ef-4266-a2d4-311bbdff052b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.435815 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8982041-31ef-4266-a2d4-311bbdff052b None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.440578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 17/32] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:51.443450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fc11-a741-468e-b841-87404df44f7c None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.443745 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fc11-a741-468e-b841-87404df44f7c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.444042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3fc11-a741-468e-b841-87404df44f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.462372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 16/33] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:09:50 2022] POST /identity/v3/projects => generated 301 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:51.465369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-6ba3fc11-a741-468e-b841-87404df44f7c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:51.466824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-6ba3fc11-a741-468e-b841-87404df44f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:51.472864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:51.559870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 18/34] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:09:51.584725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:51.658238 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e406ca5-c73e-4bdc-959b-2dac9af89c6e None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.658542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e406ca5-c73e-4bdc-959b-2dac9af89c6e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.659148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e406ca5-c73e-4bdc-959b-2dac9af89c6e None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.661074 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 19/35] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:51.669697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d0f-b207-42d6-8b77-12e1c625a309 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.670312 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d0f-b207-42d6-8b77-12e1c625a309 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.670822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d0f-b207-42d6-8b77-12e1c625a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.694870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-5a890d0f-b207-42d6-8b77-12e1c625a309 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:51.697883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-5a890d0f-b207-42d6-8b77-12e1c625a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:51.736722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703cd26-cada-4d9e-be8e-c013eb93a3bb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:51.774898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:51.788296 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:51.789193 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.789522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.789780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.810277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:51.810745 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 20/36] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:09:51.812171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efedcf0-4f03-4a2f-a51f-2455ca46a7d7 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.812521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efedcf0-4f03-4a2f-a51f-2455ca46a7d7 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.812809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efedcf0-4f03-4a2f-a51f-2455ca46a7d7 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.814436 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 21/37] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:51.819930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.820219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.820545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:51.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:51.826386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:09:51.827991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 17/38] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:09:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:51.833742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:51.846794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:51.848047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:51.917910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:51.924284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 22/39] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:51.925238 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:51.959046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40bfce-6737-444c-a79b-6e73e17cd65c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:51.994390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:51.995079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:51.995376 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:51.995661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:51.997383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:51.997680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:51.999234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.007743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 18/40] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:09:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:52.009290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.034332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cc4d9-0135-45ed-ae5a-9f8fd02e5f20 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.061444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:52.071371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.072190 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.072516 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.072803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.074307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.075780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.077482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 23/41] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:52.078351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.130472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f4c0d-99fa-405c-8fea-944768832118 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.151886 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:52.165657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.166341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.166743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.167035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.169097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.169949 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.185267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 19/42] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:09:52 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:52.185267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.204927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd8ac-12a6-4093-875d-757db2989a20 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.253089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.254036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.254331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.254616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.256024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.257610 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.259390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 24/43] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:09:52 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:52.265359 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.321675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35cdc9-d223-45c6-927e-48f1e799b1d5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.338217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:52.368588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.369281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.369577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.369866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.370601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.372378 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.397655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493997c-e21d-4f76-9247-b2c3f86e1781 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.404773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 20/44] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:09:52 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:52.447030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.447785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.448101 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.448365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.449985 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.451490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5fa673-ff35-468d-80db-2ed2b409116b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.453258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:52.459127 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:52.572415 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c3280-4a5d-4eac-a56f-40ac96556765 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:52.611653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:52.612576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:52.612865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:52.613156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] PATH_INFO: `/v3/domains/default/users/bfca2eff03f04a12a3f7eb5950753314/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:52.615210 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bfca2eff03f04a12a3f7eb5950753314, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:52.616706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9c79-64af-4c09-8be4-fc6d766b890d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:52.632022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 21/46] 10.222.0.13 () {58 vars in 1361 bytes} [Thu May 19 20:09:52 2022] PUT /identity/v3/domains/default/users/bfca2eff03f04a12a3f7eb5950753314/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:09:54.668784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b685-6641-4225-bdaf-eee7653bd776 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:54.669365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b685-6641-4225-bdaf-eee7653bd776 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:54.669365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282b685-6641-4225-bdaf-eee7653bd776 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:54.670836 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:54.677086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307433-d9e1-4861-b59a-9fa835b60583 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:54.677582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307433-d9e1-4861-b59a-9fa835b60583 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:54.677677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b307433-d9e1-4861-b59a-9fa835b60583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:54.696461 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-4b307433-d9e1-4861-b59a-9fa835b60583 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:54.698112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-4b307433-d9e1-4861-b59a-9fa835b60583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:54.718828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:54.775555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:54.799337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b685-6641-4225-bdaf-eee7653bd776 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:54.802399 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:54.890553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92cd0b-0f23-47de-9d79-2bfc9e5845f0 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:54.890846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92cd0b-0f23-47de-9d79-2bfc9e5845f0 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:54.891149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92cd0b-0f23-47de-9d79-2bfc9e5845f0 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:54.892563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 23/49] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:09:54.897498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:54.897808 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:54.898097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:54.911371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282b685-6641-4225-bdaf-eee7653bd776 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:54.915517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:54.916763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:54.946748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:54.947469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:54.947766 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:54.948052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:54.949876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:54.951456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:54.953140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 27/50] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:54.958696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:54.993525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 24/51] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:09:55.017102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:55.034493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:55.067023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed9071-9f76-4367-b29a-03049f834b9e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.103608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:55.103822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.104287 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.104617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.104903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.106309 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:55.108003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:55.125124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438924e-a778-4383-9d5b-bdf3a74141ac None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.135355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 28/52] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 20:09:54 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:09:55.164204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.165084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.165383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.165680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.178669 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:55.187818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:55.189153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:09:55.190040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:09:55.196438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:55.312185 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217c6ba-dfb1-4ea6-8675-1fe3f759f69b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.336787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:55.351860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.352682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.352977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.353263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.354490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:55.354490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:55.356646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:55.369742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 29/54] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:55.375644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:55.480995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc610ad5-fb1c-422e-8940-1f23644bf424 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.515975 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.516906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.517202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.517488 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.518716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:55.520202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:55.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 26/55] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:09:55 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:55.528160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:55.630451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e2c0b-df40-4df0-8189-7638e184e122 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.678073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.678888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.679183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.679469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.682008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:55.684067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd168f-e66e-44a1-8576-14774651d1ec None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:55.685777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:55.691512 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:55.703818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:55.798098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc869d-93cb-4a6c-ac13-75872a34afee None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:55.834506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:55.835208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:55.835503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:55.835785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:55.836668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.domain.id=default, user.id=bfca2eff03f04a12a3f7eb5950753314) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:55.836990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:55.838461 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d22da1-7467-4511-a1a1-dfcdca64b490 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:55.847448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 27/57] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 20:09:55 2022] GET /identity/v3/role_assignments?user.id=bfca2eff03f04a12a3f7eb5950753314&scope.domain.id=default&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:57.444072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0152c-30d1-449d-b88e-8fb59ba044ce None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:57.444794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0152c-30d1-449d-b88e-8fb59ba044ce None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:57.444794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0152c-30d1-449d-b88e-8fb59ba044ce None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:57.445950 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:57.452522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:57.453021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:57.453128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:57.470958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:57.472185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:57.552192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:09:57.578561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0152c-30d1-449d-b88e-8fb59ba044ce None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:57.641413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:57.694494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0152c-30d1-449d-b88e-8fb59ba044ce None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:57.732771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:57.733574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:57.733861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:57.734141 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:57.735332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:57.736841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b5e9f-e589-4adf-adb6-73fceede36c4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:57.738479 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:57.744068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:57.746351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:57.849767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47a83e-3537-4add-b6ee-93eb5faa2d60 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:57.885911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:57.887243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:57.887588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:57.887877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:57.888640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:57.890112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa019a1-17d5-46e1-bf23-4620330fe3ac None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:57.917283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:09:57 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:01.539549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f636bc-65b8-4e86-8ae5-40202f634e8e None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.539549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f636bc-65b8-4e86-8ae5-40202f634e8e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.539549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f636bc-65b8-4e86-8ae5-40202f634e8e None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.539549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:01.547012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.547917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.548487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.575417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:01.577074 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:01.641103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2791-71ba-4f1d-a5fe-9ab14454559e None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.641757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2791-71ba-4f1d-a5fe-9ab14454559e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.642347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:01.643606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a2791-71ba-4f1d-a5fe-9ab14454559e None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.645446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 34/63] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:01.653407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251deb69-de1e-4bda-b715-94f8e1433c3a None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.654275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251deb69-de1e-4bda-b715-94f8e1433c3a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.654821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251deb69-de1e-4bda-b715-94f8e1433c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.686126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 30/64] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:01.687468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-251deb69-de1e-4bda-b715-94f8e1433c3a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:01.695538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-251deb69-de1e-4bda-b715-94f8e1433c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:01.741064 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:01.845430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 35/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:01.858750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f714-57e5-4e9d-aa1d-c2eebc1799f5 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.859045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f714-57e5-4e9d-aa1d-c2eebc1799f5 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.859338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24f714-57e5-4e9d-aa1d-c2eebc1799f5 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.860785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 36/66] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:01.871775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24f714-57e5-4e9d-aa1d-c2eebc1799f5 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:01.912312 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba9b37-39ba-4b27-8342-4975bcc85073 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:01.936866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:01.953898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:01.954962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.955406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.955872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.967912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-6043683f-cef0-4b0c-a376-21972d6f6001 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:01.976381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:01.977370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:01.978229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:01.980824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 31/67] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:01.982713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee66be7-f461-4f9d-80f5-700853c83354 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:01.983226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee66be7-f461-4f9d-80f5-700853c83354 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:01.983729 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee66be7-f461-4f9d-80f5-700853c83354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:01.996439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24f714-57e5-4e9d-aa1d-c2eebc1799f5 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.003084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-4ee66be7-f461-4f9d-80f5-700853c83354 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:02.004547 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-4ee66be7-f461-4f9d-80f5-700853c83354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:02.036389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.037279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.037655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.038027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.052777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:02.056201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:02.063004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:02.063984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:02.064848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 37/68] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:02.065538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.093109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 32/69] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:02.098011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99930-6cae-42e6-8ad2-13e7dba1e639 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.098543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99930-6cae-42e6-8ad2-13e7dba1e639 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.099035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd99930-6cae-42e6-8ad2-13e7dba1e639 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.104651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 33/70] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:02.104651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99930-6cae-42e6-8ad2-13e7dba1e639 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.195569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0820a-2651-4dc5-8469-b2f5feea28c1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.208578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.223174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd99930-6cae-42e6-8ad2-13e7dba1e639 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.234868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.235556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.235854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.236138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.237022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:02.237342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.238923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b43db-7de1-4d56-a52f-0c0be080db9c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.249353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 38/71] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:02.251491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c598a9-6c91-4989-86d5-f3a961dfb2bb None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.251867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c598a9-6c91-4989-86d5-f3a961dfb2bb None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.252234 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c598a9-6c91-4989-86d5-f3a961dfb2bb None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.253817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 39/72] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:02.254960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.255268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.255555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.268633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.269366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.269664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.269959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.270735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:02.271052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.272504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.273075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:02.274294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:02.281028 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 34/73] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:02.281926 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.327182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.350629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 40/74] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:02.351509 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.401279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c8877-6756-47e9-b886-2d2921254ab4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.438161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.438983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.439279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.439598 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.450739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:02.458628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278556c-96d7-44b0-86be-9a8d2adea514 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-82003f29-23e1-4045-9062-69667ff35848 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:02.459230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:02.460324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:02.461586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:02.463148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.463244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 35/75] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:02.463244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.463244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.463244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.480464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:02.481500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:02.509840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.510765 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.511058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.511384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.525988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.527637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.529407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 41/76] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:02.530290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.554738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 36/77] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:02.555650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.589767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.638537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d191a-52d8-4660-88c5-21b7b676c96c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.674634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.675525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.675820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.676107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.677391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.678874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.680262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161042c4-07a2-4cd6-88bb-c1620eafe03e None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.685809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 42/78] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:02.685809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.711610 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.717701 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.718535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.718825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.719120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.729944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:02.738318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:02.739981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 37/79] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:02.741119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.800383 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13097e38-6015-4e81-8750-d563799e0ad2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.833585 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.835433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.836189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.836509 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.836800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.837574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:02.837890 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.839284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.844729 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c974518-fb39-47fa-891f-8434f706d738 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.848191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 43/80] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:02.849051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:02.885221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.885893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.886187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.887126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.944530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:02.955537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243e608-9ec2-44c0-90b7-65e1fb807a69 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:02.958694 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.960209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:02.995605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:02.996593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:02.996888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:02.997182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:02.998404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:02.999887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.001582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 44/81] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.002368 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.057479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.069335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 38/82] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:03.070260 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.111771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb59e8-5ccd-44e9-92d0-dbd819580fa5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.148974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.149647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.149943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.150230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.178959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.188096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16b5e69-6905-4d65-9548-a312fdb282cf None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.229045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.229276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.229384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.229384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.229384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.229384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:03.229384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.229794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.235744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.241713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 39/83] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.242576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.303369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.350518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 45/84] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:03.351384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.363483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ee99b-f1b4-4c10-9d47-cbc793cc1180 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.402779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.403763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.404055 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.404338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.405619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.407090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.408783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 40/85] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.409560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.416640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.465530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3ff8f-f9b9-48fc-9eee-113d9f6f9525 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.501685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.502289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.502583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.502872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.503605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.505140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.515518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d2eb-22f1-453d-b5d9-64851cef38e9 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.532885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 46/86] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:03 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:03.533773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.552983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.570202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.571102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.571398 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.571700 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.573021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.574952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.576848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 41/87] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.582681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.666351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08aee9-9891-4ad4-aeca-3fa02a5a92e2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.684087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.712145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.713098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.713386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.713692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.715195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.716747 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.716953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823b516-cc7b-4b4a-a641-3ecdc7bdb26d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.718711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 47/88] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.719304 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.765635 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.766576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.769959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.769959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.769959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.770239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.771936 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 42/89] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.772756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.852283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4a3b-b7a0-4d5b-b222-fc89eca85b84 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.857046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.911588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.911588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.911588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.911588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.923941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec7fb7-421c-4020-93ad-5cafe30df713 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:03.965429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:03.974468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:03.975155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:03.975448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:03.975735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:03.976621 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:03.976950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:03.978451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:03.985139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 43/90] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:03.985985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:03.995425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.002204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.124910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c277f-8ddb-4586-93da-859c80b6f03e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:04.200555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:04.220601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 48/91] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:04.222656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:04.259149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:04.259924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.260229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.260588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.345501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:04.382044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb36f1-06b1-48e8-b822-ab95b39a2318 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:04.400015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.401585 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.429831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:04.431711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.431945 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.432194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.433189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=bfca2eff03f04a12a3f7eb5950753314) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:04.433510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.436046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.456336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 49/92] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/role_assignments?user.id=bfca2eff03f04a12a3f7eb5950753314&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:04.457076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:04.702867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 44/93] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:04.708141 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607cf8f-b87c-42e8-b4dc-53b31cf650f4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:04.709756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:04.756907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:04.793549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:04.794863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.795389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.795878 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.797511 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.799228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.801227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:04.807322 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:04.848440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:04.850529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29142181-fab2-4395-8aa0-71c1b00cafec None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:04.901192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:04.903018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:04.903224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:04.903642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] PATH_INFO: `/v3/projects/8d2e1661c7e84d43846133d74d339ca2` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:04.906034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] RBAC: Authorizing `identity:get_project(project_id=8d2e1661c7e84d43846133d74d339ca2)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:04.907606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:04.909781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 45/95] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/projects/8d2e1661c7e84d43846133d74d339ca2 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:04.924568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:05.038671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f719e31-b0a0-4095-8f46-c0f3b53d1cc2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:05.076084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:05.092218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1919ef-1c0f-4669-a42a-3b723c3dc614 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:05.140064 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:05.140786 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:05.141078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:05.141362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:05.142222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:05.142550 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:05.144024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716419-2dd6-4e9f-a742-fa7d4b645100 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:05.145128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:05.145816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:05.146115 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:05.146402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:05.147264 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=8d2e1661c7e84d43846133d74d339ca2, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:05.147589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:05.149099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9514c20-f7bd-45de-a122-36394843a1ac None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:05.158774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=519d740b9c3d46f3902513905864fa5d => generated 259 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:05.159021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:04 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=8d2e1661c7e84d43846133d74d339ca2&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:06.803019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024049be-ddfb-471d-a474-663a97f97978 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:06.804094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024049be-ddfb-471d-a474-663a97f97978 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:06.804629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024049be-ddfb-471d-a474-663a97f97978 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:06.806492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:06.812235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:06.813151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:06.813660 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:06.835640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:06.837232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:06.892127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d7a-8a83-4a8e-8401-97faa82f6a62 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:06.892492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d7a-8a83-4a8e-8401-97faa82f6a62 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:06.892759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c52d7a-8a83-4a8e-8401-97faa82f6a62 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:06.895538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 53/99] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:06.899709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb867b-4c51-433e-bb82-930ac3f8d5aa None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:06.900012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb867b-4c51-433e-bb82-930ac3f8d5aa None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:06.900306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb867b-4c51-433e-bb82-930ac3f8d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:06.944719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-34fb867b-4c51-433e-bb82-930ac3f8d5aa None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:06.946098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-34fb867b-4c51-433e-bb82-930ac3f8d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:06.962470 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 47/100] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:07.005848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.081991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 54/101] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:07.083376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ccb6e-81ac-46a1-bbb8-7228993ca0b6 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.083679 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ccb6e-81ac-46a1-bbb8-7228993ca0b6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.083972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ccb6e-81ac-46a1-bbb8-7228993ca0b6 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.085643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 55/102] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:07.091490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.091791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.092080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.123140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:07.131804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:07.132750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:07.173831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:07.208514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e2ab6-3680-455c-932b-b5cbeb7ebffe None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:07.280490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 56/103] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:07.281390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.286668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:07.287511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.287805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.288092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.290486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:07.291999 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:07.296637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 48/104] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:07.307894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.429719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3cccc-34df-46ca-87a5-760357096f5f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:07.476760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:07.477457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.477747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.478036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.478770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:07.480804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:07.536275 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 49/105] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:10:07 2022] POST /identity/v3/projects => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:07.536978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.542261 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d16440-6f51-4c6f-b6d9-90ada8fd69d8 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:07.545998 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:07.625401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:07.626271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.626578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.626867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.637179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:07.643856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:07.659632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:07.660730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:07.661624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 57/106] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:07.667982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.796721 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771de2a-6f78-4944-993c-6e05929eb3a4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:07.858897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457d082-8253-4230-ac30-03e7359626d4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:07.885857 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:07.888251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.888803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.889304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.937644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:07.952426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:07.953451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:07.954328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:07.956917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 50/107] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:07.964037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:07.985445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:07.986746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:07.987063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:07.987354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:07.988154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:07.988525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:07.990440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7a986-5cb8-4ada-a797-179ca21edf10 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:08.001518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 58/108] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:08.009862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b93c-d7a3-4859-a935-5994941f7575 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.010191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b93c-d7a3-4859-a935-5994941f7575 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.010454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1b93c-d7a3-4859-a935-5994941f7575 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.013102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 59/109] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:08.014000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1b93c-d7a3-4859-a935-5994941f7575 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:08.078053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.143453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60ac7-0a67-4a6b-b454-e6e95a810b66 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:08.165588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.218832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:08.219592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.219893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.220178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.223337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:08.223666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:08.225179 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d2c5-b465-48af-99b1-80295eefa006 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:08.237202 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1b93c-d7a3-4859-a935-5994941f7575 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:08.248619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 51/110] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:08.250325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.253272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.253750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.282562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:08.287153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:08.295301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:08.296606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.297136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.298513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] PATH_INFO: `/v3/users/9847e2559d7f4fbfa9d321693133c5b3` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.321206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] RBAC: Authorizing `identity:get_user(user_id=9847e2559d7f4fbfa9d321693133c5b3)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:08.323322 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b68e61-aad2-47a9-a025-390bf4c9410c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:08.330525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 60/111] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/users/9847e2559d7f4fbfa9d321693133c5b3 => generated 289 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:08.333212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca1c3-9f68-489e-a70d-0e56d940127a None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.338477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca1c3-9f68-489e-a70d-0e56d940127a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.339042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ca1c3-9f68-489e-a70d-0e56d940127a None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.341439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 61/112] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:08.343307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca1c3-9f68-489e-a70d-0e56d940127a None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:08.444774 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 52/113] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:08.446302 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:08.538019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.540042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ca1c3-9f68-489e-a70d-0e56d940127a None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:08.562236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.583115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:08.584644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.585059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.585464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.587147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5bb4-abde-4d22-a12a-f0c49dfe0183 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:08.587347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:08.591536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcee05-6af8-4c3d-a899-faa8dd46168e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:08.593810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 62/114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:08.595396 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.595694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.595984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.614904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:08.616942 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:08.630436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:08.631482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.631996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.632740 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.722201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:08.724109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:08.724621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 63/115] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:08.725517 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:08.796225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.859618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a628a5-e3c1-456b-8637-2d9c15e7da7b None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:08.874081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 53/116] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:08.874959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:08.879845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:08.900617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:08.901948 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:08.902473 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:08.902979 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:08.915467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:08.924319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:08.925364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:08.926265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:08.928835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 64/117] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:08.930098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.001536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3cc32-59c4-42ea-aaea-5eb5d4040ce3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.040773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefe626-cd43-4bda-8c8c-3d2241d55cad None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.041825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.043212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.043508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.043922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.045129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.046618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.048506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 54/118] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.053852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.079482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.084129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.085289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.085586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.086035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.100710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-c0b2e3b7-dfd6-462c-93b0-dbf8d4a91588 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:09.111614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:09.112681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:09.113581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 65/119] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:09.116469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58ef1-9ce3-4dc4-9d77-0df86e1cc637 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.116469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58ef1-9ce3-4dc4-9d77-0df86e1cc637 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.116469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f58ef1-9ce3-4dc4-9d77-0df86e1cc637 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.118044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 66/120] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:09.118949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f58ef1-9ce3-4dc4-9d77-0df86e1cc637 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.166086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c641d-7111-4924-902f-c5a87d30a5c9 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.170970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.210572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.211807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.212310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.212829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.214049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.216353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.218611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 55/121] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.219290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.255236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f58ef1-9ce3-4dc4-9d77-0df86e1cc637 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.294082 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.294777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.295067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.295349 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.296127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:09.296474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.297916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.311755 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 67/122] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.311755 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.336562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.349177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80749b5-a796-4977-8b17-23c023a26333 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.388149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.389004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.389299 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.389584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/a7c5be151f744680932039d47817a89c/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.391636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, user_id=a7c5be151f744680932039d47817a89c, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.393154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb53f68-f411-480c-8486-9f3cdfad7259 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.408266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 56/123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:09 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/a7c5be151f744680932039d47817a89c/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:09.419942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.420390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.420814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.446603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:09.447175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.449179 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:09.454602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853466d-f7cc-41bd-b3e9-781d19d5af9e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.506282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.507243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.507801 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.508375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.509484 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:09.510052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.511981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.522260 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 68/124] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.523685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.550228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 57/125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:09.551139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.658579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94173-cfde-4eec-b159-22dcc6e78e77 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.660198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.688677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ea78d-22ef-457b-9ffb-d3e34313df8a None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.704848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.705561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.705882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.706174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.707079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=9847e2559d7f4fbfa9d321693133c5b3) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:09.707408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.709329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.717152 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 69/126] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/role_assignments?user.id=9847e2559d7f4fbfa9d321693133c5b3&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.717817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.739854 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.741838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.742381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.742886 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.744674 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.747760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.750077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:09.751202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 58/127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.752420 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.851130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddcb0f-0504-4566-a651-cccfa138061e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:09.892149 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:09.893071 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:09.893366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:09.893641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:09.895243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:09.896901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:09.899060 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 70/128] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:09.899955 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:09.900665 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a936a-55bc-4acf-bdd5-bff91c0f9af8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.023545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.024260 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.025158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.025475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.025753 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.040052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666d32a-39b0-4957-b925-f25cbd98c211 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.053549 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:10.062803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:10.064067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:10.065143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 59/129] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:10.065993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.091663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.100437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.101828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.102301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.102812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.104715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.106895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.109274 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 71/130] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:09 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:10.110521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.238257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d7e37-861d-4862-bf6e-d0881c31ef39 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.286600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc09ea-9ff1-4592-bcbf-ca37c6131ab0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.288638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.289462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.289763 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.290055 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.290849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:10.291179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.293201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.302931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 72/131] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:10.303982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.326950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.328287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.328872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.329391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.330965 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.332775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.333315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.336176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 60/132] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:10 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:10.337495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.439432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.455844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce6efc-9da0-465f-8af8-2974d95474d6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.466270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22dcc-2c42-4da4-b80b-9f623811ea39 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.495310 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.496893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.497190 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.497483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/bfca2eff03f04a12a3f7eb5950753314/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.499769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, user_id=bfca2eff03f04a12a3f7eb5950753314, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.501294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3df20f-284e-4c80-9583-4ed942f33b04 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.507909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.509348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.509766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.510109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.511818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.514010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.516266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 61/133] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:10 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:10.517020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.519936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 73/134] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:10 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/bfca2eff03f04a12a3f7eb5950753314/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:10.526843 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88684-a03b-41e9-83b9-a23213559cf6 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.527505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88684-a03b-41e9-83b9-a23213559cf6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.528116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe88684-a03b-41e9-83b9-a23213559cf6 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.529846 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 74/135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:10.530977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88684-a03b-41e9-83b9-a23213559cf6 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.743046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea90d0-ed4b-4d2e-94d8-0cf8fa093022 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.746156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.788637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe88684-a03b-41e9-83b9-a23213559cf6 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:10.799654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.800602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.800902 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.801216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/a7c5be151f744680932039d47817a89c/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.803733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, user_id=a7c5be151f744680932039d47817a89c, role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.805248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62810d06-15ca-4658-980a-0ddc06a3d3c2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.825848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 62/136] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:10 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/a7c5be151f744680932039d47817a89c/roles/519d740b9c3d46f3902513905864fa5d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:10.827716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.828067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.828368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.833647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:10.835174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:10.835745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:10.837250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:10.837923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] PATH_INFO: `/v3/projects/8d2e1661c7e84d43846133d74d339ca2` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:10.840005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=8d2e1661c7e84d43846133d74d339ca2)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:10.842017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:10.845207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 75/137] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:10 2022] GET /identity/v3/projects/8d2e1661c7e84d43846133d74d339ca2 => generated 312 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:10.853401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:10.855007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:10.856583 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:10.947317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 63/138] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:11.004378 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:11.080810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6d7d5-086e-42b5-91f0-38beb5468fc5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:11.127900 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:11.128963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:11.129260 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:11.129549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] PATH_INFO: `/v3/projects/8d2e1661c7e84d43846133d74d339ca2/users/a7c5be151f744680932039d47817a89c/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:11.133571 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] RBAC: Authorizing `identity:create_grant(project_id=8d2e1661c7e84d43846133d74d339ca2, user_id=a7c5be151f744680932039d47817a89c, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:11.135118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:11.161373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 76/139] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:10 2022] PUT /identity/v3/projects/8d2e1661c7e84d43846133d74d339ca2/users/a7c5be151f744680932039d47817a89c/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:11.196234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db264e89-6886-45c5-b933-bd0963438b2a None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:11.252820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:11.252820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:11.252820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:11.252820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:11.253268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:11.253852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b1437-6eca-47f9-94e0-566907d6110c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:11.256026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 64/140] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:11.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:11.276927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:11.428677 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f13207-5191-43c0-b5f6-5c4c1c63a21c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:11.491281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:11.491281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:11.491609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:11.491868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:11.492650 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:11.494136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf35f6-3c5d-4508-bb47-185579219213 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:11.522629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 77/141] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:11 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:12.618630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a468b-01ad-4738-940e-f5685bf8adfd None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:12.619162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a468b-01ad-4738-940e-f5685bf8adfd None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:12.619450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a468b-01ad-4738-940e-f5685bf8adfd None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:12.621644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 65/142] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:12.629222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:12.629222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:12.629222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:12.650591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:12.651882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:12.721969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:12.730968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 78/143] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:12.759175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a468b-01ad-4738-940e-f5685bf8adfd None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:12.763009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:12.913999 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a468b-01ad-4738-940e-f5685bf8adfd None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:12.963794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:12.965386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:12.965898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:12.966411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:12.990053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:12.999138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:13.000269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:13.001228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 66/144] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:13.006781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:13.156463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e685342-0823-4970-acf5-e7724c4a53ef None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:13.204776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:13.205483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:13.205778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:13.206067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:13.206856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:13.207175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:13.208634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:13.218928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 79/145] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:13.225525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:13.319412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:13.378305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b7151-f890-4ab7-8a11-97dabed316b7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:13.423388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:13.424287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:13.424624 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:13.424911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:13.426135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:13.427619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc935e5-3d77-45ab-8d1b-204fdc43ce4f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:13.434402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 67/146] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:13 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:13.439622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:13.514050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:13.566079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d964315-6d68-480b-91fe-32dfb1c38f6c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:13.618761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:13.619433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:13.619722 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:13.620002 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:13.679503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae721-20dc-49d8-b299-3d952ba3af68 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:13.680067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae721-20dc-49d8-b299-3d952ba3af68 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:13.680575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fae721-20dc-49d8-b299-3d952ba3af68 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:13.683612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 68/147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:13.690592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:13.691128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:13.691652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:13.753578 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:13.754919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:13.774754 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:13.776182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:13.844194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:13.906236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 80/148] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:13.911918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:13.958824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 69/149] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:14.017545 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:14.050034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36243aab-71a0-4b98-a293-6c1e5e348b65 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:14.103930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:14.105405 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.106196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.106196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.107208 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:14.108753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:14.110538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 81/150] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:13 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:14.116154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:14.168508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:14.226087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:14.281425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f00e9-7738-4ff3-8697-f99cadfc7f2f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:14.326254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694d09c-029f-47ab-bde5-fdb8a4e8c96f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:14.334703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:14.335393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.335673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.335954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.340267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:14.340267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:14.343000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:14.356998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 82/151] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:14 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:14.386756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:14.386756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.386756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.386756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.410330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:14.418941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:14.420905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:14.424634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 70/152] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:14.426973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:14.562956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:14.580012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a93b8-fd41-4b2c-9d81-2f118854c496 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:14.641055 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:14.641769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.642058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.642339 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.643176 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:14.643494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:14.645133 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:14.656855 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 83/153] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:14.662782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:14.817232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e14461-5861-4366-a0c7-7138ff670c27 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:14.847425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:14.867952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:14.868891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:14.869185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:14.869469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] PATH_INFO: `/v3/users/9847e2559d7f4fbfa9d321693133c5b3` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:14.870761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] RBAC: Authorizing `identity:get_user(user_id=9847e2559d7f4fbfa9d321693133c5b3)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:14.872239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390c14b-a953-4478-8627-6909d1dec8d4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:14.874294 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 71/154] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:14 2022] GET /identity/v3/users/9847e2559d7f4fbfa9d321693133c5b3 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:14.881530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:15.015644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1b7c-80e2-45f1-ac3f-94e68bd0d448 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.016022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1b7c-80e2-45f1-ac3f-94e68bd0d448 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.016216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac1b7c-80e2-45f1-ac3f-94e68bd0d448 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.017650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 72/155] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:15.020187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45279cef-307a-44c0-aaa3-0ca6907f41e8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:15.029975 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ff09-57a6-478a-a1a0-4dff64832d75 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.030325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ff09-57a6-478a-a1a0-4dff64832d75 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.030544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ff09-57a6-478a-a1a0-4dff64832d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.068279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-5945ff09-57a6-478a-a1a0-4dff64832d75 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:15.069499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5945ff09-57a6-478a-a1a0-4dff64832d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:15.073090 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:15.074302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.074596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.074872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.076130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:15.077651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:15.079391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 84/156] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:14 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:15.089786 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:15.142057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:15.254383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 73/157] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:15.255744 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be245a-da68-4b18-9a14-7ee726f35e1f None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.256040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be245a-da68-4b18-9a14-7ee726f35e1f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.256322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be245a-da68-4b18-9a14-7ee726f35e1f None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.257638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 74/158] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:15.272133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aefd-3f39-465d-acad-b4c32b6d46cd None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.272495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aefd-3f39-465d-acad-b4c32b6d46cd None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.272735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aefd-3f39-465d-acad-b4c32b6d46cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.313772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d712aefd-3f39-465d-acad-b4c32b6d46cd None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:15.315108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d712aefd-3f39-465d-acad-b4c32b6d46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:15.337951 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c6a0b-45a3-4b31-b0b9-2e19aa35f7d4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:15.424642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:15.425503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.425793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.426079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/9847e2559d7f4fbfa9d321693133c5b3/roles/c0f70d47fd324f5ba11fc34d35c801b5` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.432139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, user_id=9847e2559d7f4fbfa9d321693133c5b3, role_id=c0f70d47fd324f5ba11fc34d35c801b5)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:15.434361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:15.455289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 85/159] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:15 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/users/9847e2559d7f4fbfa9d321693133c5b3/roles/c0f70d47fd324f5ba11fc34d35c801b5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:10:15.456355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:15.474736 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 75/160] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:15.476389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcb9b-d169-4313-8cc5-1049b698ee51 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.476752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcb9b-d169-4313-8cc5-1049b698ee51 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.476990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bcb9b-d169-4313-8cc5-1049b698ee51 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.478785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 76/161] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:15.490724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.491070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.491288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.523097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:15.524450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:15.601092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:15.657584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 77/162] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:15.658535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:15.693931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:15.754695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e48ce5-3963-4fd8-a41e-950277295371 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:15.848065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a49889-c746-4fcd-9984-3fddb75253e6 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:15.865160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:15.865994 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.866288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.866575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.890119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:15.897401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:15.897944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:15.898410 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:15.898845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:15.905681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:15.906943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:15.908001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 86/163] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:15.916291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:15.917928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:15.927108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:15.928326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:15.929416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:15.932366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 78/164] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:15.934126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.009229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.125319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03a564-6588-4922-b935-5de59f436e73 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.161929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9313520-016f-4f95-a6fb-fcadf613a2b9 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.212193 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.212960 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.213247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.213523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.214952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:16.215441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.216896 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:16.231899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.232770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.233058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.233338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.237614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 79/165] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:16.238481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.253260 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.259780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:16.275357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:16.276460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:16.277919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 87/166] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:16.279265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.413752 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f492783-c0dd-4418-aa6d-b52ee4ef57c0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.454376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.455314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.455600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.455942 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] PATH_INFO: `/v3/users/bfca2eff03f04a12a3f7eb5950753314` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.457621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] RBAC: Authorizing `identity:get_user(user_id=bfca2eff03f04a12a3f7eb5950753314)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.459329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:16.461219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 88/167] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/users/bfca2eff03f04a12a3f7eb5950753314 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:16.462187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.473761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.477904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d0ec-12e3-4725-96a6-45059d68a9ec None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.546017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.553018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.553726 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.554012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.554290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.555079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:16.555393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.557240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:16.576717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 80/168] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:16.577429 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.589951 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5c22c-fddc-4c60-8a4e-9d38ca2f4317 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.632791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.633487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.633774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.634224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.635008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:16.635323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.638012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:16.648186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 89/169] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:16.650028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.752608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dededa-665d-4e3b-b053-969375d3620f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.786763 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b5e7-7f5d-49ad-ab4f-564244d9c524 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:16.794468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.795137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.795425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.795699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.825564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.847239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:16.853178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:16.854080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:16.854365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:16.854643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:16.857050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.860979 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:16.863619 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 90/170] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:16.864429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:16.966631 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:16.969240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.133151 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a17539-763a-4468-895a-bcb6b23b0b7e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.146505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 81/171] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:17.147912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.181527 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.182583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.183347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.183639 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] PATH_INFO: `/v3/users/a7c5be151f744680932039d47817a89c` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.185318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] RBAC: Authorizing `identity:get_user(user_id=a7c5be151f744680932039d47817a89c)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.187539 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f084f86b-d7f2-4713-9bfe-a4c168187e2d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 91/172] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:16 2022] GET /identity/v3/users/a7c5be151f744680932039d47817a89c => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c383-27c1-451d-a87d-bf73f8212225 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c383-27c1-451d-a87d-bf73f8212225 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3c383-27c1-451d-a87d-bf73f8212225 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 92/173] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:17.195697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb309-3d8b-461f-a727-76ec4b5648cc None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.199300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb309-3d8b-461f-a727-76ec4b5648cc None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.199300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcefb309-3d8b-461f-a727-76ec4b5648cc None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.199300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 93/174] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:17.199300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefb309-3d8b-461f-a727-76ec4b5648cc None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.241505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.285379 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.285742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb9d83-b322-4b4e-afe0-85bc2bd3ec3c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.326669 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.327052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.327337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.327617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.328440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcefb309-3d8b-461f-a727-76ec4b5648cc None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.371507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.372389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.372776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.373059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.374294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.375779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.382446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 94/175] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:17.382446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.425419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.426953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c1915-7c76-429c-ac38-2cd48c0592b3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.504853 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.510034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf80e7-ce03-4248-aa14-89dc1905d683 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.547538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.548230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.548545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.548831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.551593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.552944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 82/176] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:17.554782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dbca7-ed50-4fe0-ac46-ede2e0133a6e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.555892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dbca7-ed50-4fe0-ac46-ede2e0133a6e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.556877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3dbca7-ed50-4fe0-ac46-ede2e0133a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.575845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-3e3dbca7-ed50-4fe0-ac46-ede2e0133a6e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:17.577096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-3e3dbca7-ed50-4fe0-ac46-ede2e0133a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:17.631316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.632959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.660436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 83/177] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:17.660436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f0b9-0755-44fb-989a-79a079819306 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.660436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f0b9-0755-44fb-989a-79a079819306 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.660436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f0b9-0755-44fb-989a-79a079819306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.674945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-8a96f0b9-0755-44fb-989a-79a079819306 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:17.676324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-8a96f0b9-0755-44fb-989a-79a079819306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:17.773801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 84/178] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:17.775464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ee99-5bda-4518-a02b-d2f7f9625696 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.775906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ee99-5bda-4518-a02b-d2f7f9625696 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.776325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45ee99-5bda-4518-a02b-d2f7f9625696 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.778368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 85/179] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:17.779213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45ee99-5bda-4518-a02b-d2f7f9625696 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.780619 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 95/180] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:17.785217 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.813046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.823556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:17.898517 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd4eb3-69e2-4afb-bf0d-e03fb2697176 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.906036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45ee99-5bda-4518-a02b-d2f7f9625696 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:17.937499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.938303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.938565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.938839 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] PATH_INFO: `/v3/projects/8d2e1661c7e84d43846133d74d339ca2` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.940202 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] RBAC: Authorizing `identity:get_project(project_id=8d2e1661c7e84d43846133d74d339ca2)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.941689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.948244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 96/181] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/projects/8d2e1661c7e84d43846133d74d339ca2 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:17.948244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:17.949525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:17.950279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:17.950746 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:17.951151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:17.952051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=bfca2eff03f04a12a3f7eb5950753314) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:17.952480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:17.954033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:17.961257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 86/182] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/role_assignments?user.id=bfca2eff03f04a12a3f7eb5950753314&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:17.962144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.071041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.091782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60232f-7cf0-4862-a4f9-828e5a6a50ff None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.110092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206efb3-8d20-4fc4-80d2-e49614c46dc2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.112561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.131831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.132797 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.133094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.133380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.134862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.136976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e563-174f-425c-aa8f-78eb0fc2c580 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.143569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 87/183] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:18.143569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.143569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.143569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.161599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:18.162816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:18.165685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.166622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.166917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.167201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.168634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.170153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.172235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 97/184] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:18.177471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.257704 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 88/185] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:18.257704 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.336227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49168fc6-86ae-4372-a472-e961e0432d16 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.370047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.375142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.375970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.376267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.376584 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.387568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:18.390435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d206e9e-681d-4f0f-9cd2-d4342737868c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:18.395833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:18.397024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:18.397905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:18.402771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 98/186] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:18.403394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.433903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.434582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.434871 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.435152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.436002 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=8d2e1661c7e84d43846133d74d339ca2, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:18.436324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.437754 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.437911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.445226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 89/187] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=8d2e1661c7e84d43846133d74d339ca2&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:18.445849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.531104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466df67a-00c7-46ff-985d-43b089cb2d35 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.572906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687730c0-7795-4714-bb30-b451729d276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.573740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730c0-7795-4714-bb30-b451729d276b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.574040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730c0-7795-4714-bb30-b451729d276b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.574345 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687730c0-7795-4714-bb30-b451729d276b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.575290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730c0-7795-4714-bb30-b451729d276b None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=a7c5be151f744680932039d47817a89c) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:18.575621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730c0-7795-4714-bb30-b451729d276b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.577154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687730c0-7795-4714-bb30-b451729d276b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.583759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 99/188] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/role_assignments?user.id=a7c5be151f744680932039d47817a89c&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=519d740b9c3d46f3902513905864fa5d => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:18.584671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687730c0-7795-4714-bb30-b451729d276b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.585134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a5a54-bc2d-4c31-bb0c-f56809d5814b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.648100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.649700 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.650161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.650446 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.651351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.653787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.681252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 90/189] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:18 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:18.682199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.686528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.742288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:18.753692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687730c0-7795-4714-bb30-b451729d276b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.812883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.814323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.814801 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.815251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.831086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-010c8306-86f6-41ab-843b-0e9a5ea3383c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:18.842641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:18.843912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:18.844922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:18.846783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 100/190] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:18.847828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f0a3-cbb1-4c4e-937c-91d00f2c8c95 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.848056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f0a3-cbb1-4c4e-937c-91d00f2c8c95 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.849789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d4f0a3-cbb1-4c4e-937c-91d00f2c8c95 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.850173 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 101/191] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:18.853632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f0a3-cbb1-4c4e-937c-91d00f2c8c95 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:18.856758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bb15a-33a6-4360-bf3a-a9a170dfa097 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:18.920568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:18.921279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.921572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.921860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.922665 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:18.922991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:18.925785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892de60-6bc6-459d-a136-93adb3f4d9a8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:18.937067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 91/192] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:18.939158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:18.939453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:18.939738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:18.961479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:18.962681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:19.014688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d4f0a3-cbb1-4c4e-937c-91d00f2c8c95 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.047091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.061697 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 92/193] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:19.062592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.064032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.064254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.066870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.066870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.066870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:19.066870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.067877 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.079864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 102/194] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:19.086027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.111737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.215621 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0065-5d3e-404d-9071-da76e13a135d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.223395 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99641a-25de-42de-8b2b-94eaf3ec675b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.254078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.254958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.255242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.255520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.267916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.269227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.269595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.269979 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.271900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.273432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.275618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 93/195] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:19.276508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.288666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.290725 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.294224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 103/196] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:19.295523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.346756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.396472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0424e-fd6b-4772-ab2a-d013c1bc13f8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.404823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.429935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f37f3-27f6-4658-8e2a-aaf981fbb7aa None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.434383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.435494 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.435997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.436509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.451612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:19.459951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:19.461106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:19.462051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:19.465233 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 94/197] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:19.466473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.495652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.496844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.497328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.497810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.579331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.580885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.596014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5370b4-7c72-4304-812d-0b6492e1e4f2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.613401 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.636864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.637640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.638003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.638355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.686217 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.722712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.723288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 104/198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:19.723680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.724680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.844304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 95/199] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:19.844304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.852895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990c024-831f-4fc6-a82d-531c2ac8212e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.861281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.899405 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.900380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.900938 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.901421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.902952 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:19.903451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.905297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:19.915769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 105/200] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:19.917373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:19.954274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cbb9f-9be9-48b5-b062-382700a07801 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:19.956704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:19.994496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:19.995992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:19.996290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:19.996687 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:19.998761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:19.999595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.006256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 96/201] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.006256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.037534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a2104-396f-471f-843b-0cd6b1176941 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.076994 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.078285 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.078868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.079364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.080884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.082579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.089863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 106/202] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:19 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.091702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.096835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:20.120544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082fc51-45ef-49ac-a26a-d729ad0819f7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.161865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.162678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.162968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.163249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] PATH_INFO: `/v3/users/9847e2559d7f4fbfa9d321693133c5b3` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.164913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] RBAC: Authorizing `identity:get_user(user_id=9847e2559d7f4fbfa9d321693133c5b3)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.166425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.170260 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 97/203] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/users/9847e2559d7f4fbfa9d321693133c5b3 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.171041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.217165 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598e6af-c9f9-4a95-abd4-dbd1b4875b7d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.220804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:20.256237 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.257805 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.258301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.258789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.259849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=5144e4689f8246b0b8f38f83f0541692) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:20.260458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.263237 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.277396 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 107/204] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/role_assignments?user.id=5144e4689f8246b0b8f38f83f0541692&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.279084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.299553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1c9d3-fb28-4827-8fed-097ed2ae2871 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.340431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.341216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.341505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.341785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.342779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=5144e4689f8246b0b8f38f83f0541692) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:20.343108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.344644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.351155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 98/205] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/role_assignments?user.id=5144e4689f8246b0b8f38f83f0541692&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=519d740b9c3d46f3902513905864fa5d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.442165 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6885f7-61c0-4aba-9a26-88e0aa905bb6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.483549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.484687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.485156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.485612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.529074 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:20.568974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.570772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.705294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 108/206] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:20.712426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:20.740842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:20.838918 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a2c58-6409-4d68-a87b-3ab33eea6bc2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:20.877522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:20.878407 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:20.878699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:20.878980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:20.880239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:20.881745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93ff9b-20f7-4bf0-afc3-b77a79c00ed9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:20.883809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 99/207] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:20.889965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:21.006189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:21.052093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8d54-e22e-40f5-9ff5-cd94bc5e9637 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:21.104269 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:21.105328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.105742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.106130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.107194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, scope.project.id=18c9cf67e140413ea746a42df18a6f98, user.id=9847e2559d7f4fbfa9d321693133c5b3) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:21.107633 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:21.109381 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b648cff-8d77-4029-9bfc-4d726056b18f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:21.125939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 109/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:20 2022] GET /identity/v3/role_assignments?user.id=9847e2559d7f4fbfa9d321693133c5b3&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:21.545112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167cfa7-1272-424d-9fb1-b362862cb250 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.545666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167cfa7-1272-424d-9fb1-b362862cb250 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.545666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167cfa7-1272-424d-9fb1-b362862cb250 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.547065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 100/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:21.552826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.553360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.553521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.572330 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:21.574193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:21.623249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7e4fe-46e8-4ddd-81aa-e1fd553a252d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.623590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7e4fe-46e8-4ddd-81aa-e1fd553a252d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.623833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7e4fe-46e8-4ddd-81aa-e1fd553a252d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.625348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 101/210] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:21.631310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.631652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.631880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.655329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:21.657059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:21.668971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 110/211] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:21.697925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:21.771221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 102/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:21.782872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:21.795758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:21.811056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:21.829448 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecabb78-5a3a-450e-8c9f-cbb3000da302 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:21.868682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:21.869498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.869787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.870072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.871327 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:21.872883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:21.879229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 111/213] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:21.880632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:21.905450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b48c23-64e0-445b-80df-091386f71b6c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:21.946188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:21.947030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:21.947430 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:21.947768 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:21.961177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:21.972464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:21.973550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:21.974465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 103/214] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:21.981479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:22.002554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a578ba-d917-47c9-9cc2-c6839a392c54 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:22.039223 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:22.052586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:22.053354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:22.053791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:22.054171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:22.055059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:22.057517 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:22.079814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:22.086685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 112/215] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:21 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:22.124188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6076-f284-4692-bc82-c890e0fed225 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:22.168991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:22.169738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:22.170032 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:22.170317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:22.171101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:22.171462 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:22.172934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:22.188557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 104/216] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:22.193566 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:22.321225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a55b0-dec0-4b92-97d6-f2356ee1f4af None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:22.361274 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:22.362563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:22.363049 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:22.363538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] PATH_INFO: `/v3/users/713d1c6f09974e31b89121156e431c1f` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:22.382253 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] RBAC: Authorizing `identity:get_user(user_id=713d1c6f09974e31b89121156e431c1f)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:22.383753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:22.385578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 113/217] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:22 2022] GET /identity/v3/users/713d1c6f09974e31b89121156e431c1f => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:22.392816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:22.523448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f25-aab4-4600-8af8-46253b454e03 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:22.563221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:22.564030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:22.564290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:22.564621 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:22.568091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:22.644764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:22.646845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e428608-c216-4d95-9843-ab177bcf38a8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:22.767477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 105/218] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:22.773147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:22.863012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:22.949556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93f7dc-164a-468f-8c28-2320ef939382 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:22.994798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e269f-3f84-4670-a7db-f88f91edcfdc None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:22.995164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e269f-3f84-4670-a7db-f88f91edcfdc None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:22.995431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e269f-3f84-4670-a7db-f88f91edcfdc None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:22.997191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 106/219] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:23.001391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.002283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.002490 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.002597 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.002776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.002879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.003060 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.004075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:23.005720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:23.007468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 114/220] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:22 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:23.013902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.029683 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:23.030985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:23.063964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:23.109359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 107/221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:23.132792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.214489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb7e24-83e6-4ee9-8e13-0b816190fe5d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.230462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:23.256193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.257124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.257421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.257706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.258576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=713d1c6f09974e31b89121156e431c1f) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:23.258896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:23.260548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf5485-4a9d-49c4-b4dd-263dffe21cbd None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:23.267746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 115/222] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/role_assignments?user.id=713d1c6f09974e31b89121156e431c1f&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:23.273986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0368-e966-4dc6-8552-0cbe86054936 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.274279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0368-e966-4dc6-8552-0cbe86054936 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.274555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e0368-e966-4dc6-8552-0cbe86054936 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.284513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 116/223] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:23.289970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.290261 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.290637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.294175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d4f0a-538d-439a-ac24-421cbfa51fb8 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.333460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:23.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:23.363630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.364488 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.364781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.365062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.387172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:23.395362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:23.396455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:23.397345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:23.400168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 108/224] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:23.405274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.454858 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:23.516930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 117/225] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:23.540250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.576241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f02ee-9088-4326-b349-c59a41cd4a73 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.617615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.618615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.619278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.619749 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.621548 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:23.622050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:23.623626 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:23.634484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 109/226] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:23.642020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.766936 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:23.785027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac814a7b-7e02-450f-b405-02bfa6aa74c4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.814463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f37958-9bd8-4b28-ba7d-13cd21d259ee None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:23.826670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.827938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.828498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.828981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.829971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:23.832223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:23.833971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:23.836110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 110/227] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:23.843491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:23.896363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:23.897210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:23.897494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:23.897772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:23.918142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:23.927529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:23.928689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:23.929573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:23.934181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 118/228] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:23.941528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:24.028988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd739-49a7-487a-83c5-12b2fb2990f6 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.099611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e3d7a-b13e-4fab-b0c5-a6d3631e7272 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.140760 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.142003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.142579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.143068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.144094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:24.144218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.144763 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.145041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.145337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.145625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.147224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.149129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.150904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.158083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 119/229] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:24.158229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 111/230] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:23 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:24.163654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:24.164826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:24.200512 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:24.260775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:24.282819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e4c3-c295-45c6-b154-78df7b11c5df None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.321451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.322702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.323221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.323823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.325477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.327146 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.329038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 120/231] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:24 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:24.337657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:24.445708 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6a1a-8f7e-4296-895d-2c66ed9b3570 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.483869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.485196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.485691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.486195 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/5144e4689f8246b0b8f38f83f0541692/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.488630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, user_id=5144e4689f8246b0b8f38f83f0541692, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.490659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.508150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 112/232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:24 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/5144e4689f8246b0b8f38f83f0541692/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:24.564620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:24.571471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846825d1-34f0-4cf0-a660-f8dc89f33c5f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.614388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.615279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.615570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.615851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.617102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.618668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be61848-68e8-41c4-b7dd-8e1245afa590 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.620915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 121/233] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:24 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:24.626690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:24.742197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:24.764188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c092cbb-60c6-47c0-a1b1-eddc7e1008f2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:24.802297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:24.803366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:24.803659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:24.803946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/5144e4689f8246b0b8f38f83f0541692/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:24.806795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, user_id=5144e4689f8246b0b8f38f83f0541692, role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:24.808763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ab1f4-6900-4960-a7bd-9344e4f3d0d5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:24.827737 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 113/234] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:24 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/5144e4689f8246b0b8f38f83f0541692/roles/519d740b9c3d46f3902513905864fa5d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:25.204264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e2d1-b372-446c-862e-901f936e360c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.204816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e2d1-b372-446c-862e-901f936e360c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.204899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37e2d1-b372-446c-862e-901f936e360c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.206256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 122/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:25.211887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.212496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.212988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.233050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:25.234612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:25.248682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1af7e-a634-4654-b711-7f8706f91430 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.248996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1af7e-a634-4654-b711-7f8706f91430 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.249262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1af7e-a634-4654-b711-7f8706f91430 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.250665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 123/236] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:25.256245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.256571 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.256857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.278864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:25.280086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:25.358125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 114/237] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:25.367845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 124/238] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:25.392319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:25.397823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:25.484725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:25.549569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:25.559669 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e57bf7-dec4-4d9b-b16f-a170908b8a42 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:25.571214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7124d79-c379-48e1-9619-4cbb085ba1f7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:25.605394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:25.606627 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.607694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.608199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.621795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:25.631731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:25.632830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:25.633715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:25.637474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 125/239] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:25.646425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:25.662078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:25.662893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.663177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.663454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.666103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:25.668123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:25.671251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 115/240] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:25.677396 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:25.838202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105e8ea-a451-42f6-b2e4-116b903cb676 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:25.874408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97546b3-9687-4eb8-a2ce-a833914a6ded None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:25.888812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:25.889904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.890420 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.890916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.891938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:25.896347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a64d17-9f82-47f7-a92a-c0b57346b376 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:25.907574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:25.927533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:25.927788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 116/241] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:25 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:10:25.936433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.936644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585c620-0e3d-40be-993b-9a751922339a None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.936644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585c620-0e3d-40be-993b-9a751922339a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.936644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585c620-0e3d-40be-993b-9a751922339a None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.937178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 117/242] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:25.938145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.938668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.939143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:25.939491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:25.939786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:25.940496 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:25.941029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:25.942700 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:25.967143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 126/243] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:25.969336 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:25.971412 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:25.972484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:26.024546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.080779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 118/244] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:26.101336 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.211096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960aaa0-1b43-49d1-b7e5-38066ad3cb33 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.238255 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf142040-6748-4379-a709-38dc8bf5411d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.253354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.254343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.254672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.255008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] PATH_INFO: `/v3/users/4b5af7af380d45c083e789876db4f1b2` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.270419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.273157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] RBAC: Authorizing `identity:get_user(user_id=4b5af7af380d45c083e789876db4f1b2)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.274673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.277623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 127/245] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:25 2022] GET /identity/v3/users/4b5af7af380d45c083e789876db4f1b2 => generated 297 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:26.285262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.286092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.286381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.286663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.288147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.299384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:26.310366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:26.311472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:26.317310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 119/246] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:26.319937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.397746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.443570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bd409-8953-401f-b88a-fe146f85d41d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.452188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c85ff-eb4f-4afd-9799-1de1657acde4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.499343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.500026 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.500320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.501090 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.505069 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.505760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.506051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.506335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.507243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:26.507566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.509943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.524577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 120/247] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:26.529443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.587497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.597610 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.599991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.657386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293734a9-bd9d-4af4-8b30-30df8caac9d7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.711433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.713171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.713691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.714755 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] PATH_INFO: `/v3/users/713d1c6f09974e31b89121156e431c1f` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.717106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.718153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] RBAC: Authorizing `identity:get_user(user_id=713d1c6f09974e31b89121156e431c1f)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.721332 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.725272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 121/248] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/users/713d1c6f09974e31b89121156e431c1f => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:26.732012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.757739 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 128/249] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:26.764851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.882756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc3672-993c-4e7c-93f4-825481fe7306 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.908913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:26.932457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef01014-4b49-4ab2-9351-261afe9c2e89 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:26.936447 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.937711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.938236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.938838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.944573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.946497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.948743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 122/250] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:26.963546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:26.975394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:26.976384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:26.976779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:26.977138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:26.978640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:26.984473 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:26.984473 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 129/251] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:26.990363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:27.100949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:27.175243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dac344-32e5-4236-95b2-8e8ab180de49 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:27.253581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:27.255652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.256191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.256671 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/713d1c6f09974e31b89121156e431c1f/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.258967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, user_id=713d1c6f09974e31b89121156e431c1f, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:27.260627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb629c0-f1ed-412d-a189-4bcc8700ce4d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:27.274642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320199a-0fe1-441e-90f7-0ae426ea8d4c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:27.286670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 123/252] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:26 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/713d1c6f09974e31b89121156e431c1f/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:27.347347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:27.348063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.348354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.348664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.349542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=4b5af7af380d45c083e789876db4f1b2) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:27.349866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:27.351409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f5014-0791-4ae7-8e62-753c0495ed7b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:27.361765 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 130/253] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:26 2022] GET /identity/v3/role_assignments?user.id=4b5af7af380d45c083e789876db4f1b2&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 259 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:27.831531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf799cc-52ee-4cdc-85f4-393ca3cd6219 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.832666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf799cc-52ee-4cdc-85f4-393ca3cd6219 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.833212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf799cc-52ee-4cdc-85f4-393ca3cd6219 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.835184 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 124/254] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:27.840554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9554be-dbf7-4eec-86e5-9b0dafec0ccc None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.840855 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9554be-dbf7-4eec-86e5-9b0dafec0ccc None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.841218 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9554be-dbf7-4eec-86e5-9b0dafec0ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.863104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-8b9554be-dbf7-4eec-86e5-9b0dafec0ccc None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:27.864315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-8b9554be-dbf7-4eec-86e5-9b0dafec0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:27.955296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 131/255] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:27.984993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5f83-605c-43a1-bd5a-a13cc295d028 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.985657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5f83-605c-43a1-bd5a-a13cc295d028 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.986174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba5f83-605c-43a1-bd5a-a13cc295d028 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:27.988082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 125/256] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:27.998565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:27.998862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:27.999153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.002359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba5f83-605c-43a1-bd5a-a13cc295d028 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.026309 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:28.027826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:28.102625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:28.106073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:28.218884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 132/257] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:28.245940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba5f83-605c-43a1-bd5a-a13cc295d028 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:28.254248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.346214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:28.347070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.347364 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.347653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.368010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:28.380788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:28.381842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:28.382730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 126/258] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:28.394902 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.472160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cf70f-f0c7-456c-8bbd-b0d3aa6ea135 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:28.516734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:28.517985 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.518502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.518985 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.537308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:28.546410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:28.547455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:28.548362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:28.552170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 133/259] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:28.557893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.591789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:28.701153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3d9e0-5fe8-499e-be80-c74b6686d57a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:28.706601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:28.781201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7da21-ded8-47a6-86ec-5d5b85ade1b7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:28.799899 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:28.800617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.800907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.801199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.801973 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:28.802289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:28.803689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:28.823715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 127/260] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:28.841587 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.923307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:28.923307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:28.923307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:28.923307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:28.923759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:28.923759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:28.923759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:28.942936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 134/261] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:28.947868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:28.998751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26344258-6d68-4e5b-95ea-a4622d85219e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:29.049964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:29.051288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.051950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.052482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.054218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:29.057056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:29.060210 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 128/262] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:29.068818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:29.118632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:29.154938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:29.159630 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328787c-a3b2-453a-9f46-32b8bcb9ecf0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:29.267461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:29.268287 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.268602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.268885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] PATH_INFO: `/v3/users/5144e4689f8246b0b8f38f83f0541692` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.270768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] RBAC: Authorizing `identity:get_user(user_id=5144e4689f8246b0b8f38f83f0541692)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:29.272093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:29.288991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 135/263] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:28 2022] GET /identity/v3/users/5144e4689f8246b0b8f38f83f0541692 => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:29.290003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:29.361083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38aab0d-8bbf-4a7f-bc68-dd09f9b3d931 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:29.425639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:29.426824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.427320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.427804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.508803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92ab3-42fb-4671-afa1-717f4b7bdab8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:29.523142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:29.525081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:29.578023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:29.578023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.578023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.578023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.581652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:29.605494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:29.674347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 129/264] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:29.683623 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:29.734655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:29.736573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9070ea-0e62-4c19-8501-60819b0e859c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:29.862759 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10126a-d7e0-4482-98f4-a33a1a3f52cc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:29.916966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:29.919125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.919706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.920336 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.924083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 136/265] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:29.924814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:29.925496 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4dac8-5d1d-476c-b650-0db7e55822ec None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.925797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4dac8-5d1d-476c-b650-0db7e55822ec None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.926085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4dac8-5d1d-476c-b650-0db7e55822ec None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.927272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ef452-bf9a-4929-bd02-dba8559daae8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:29.927388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 137/266] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:29.929738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 130/267] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:29 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:29.933526 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4dac8-5d1d-476c-b650-0db7e55822ec None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:29.940743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:29.941367 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:29.941870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:29.947500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:29.971782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:29.973497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:30.003169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.050568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 131/268] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:30.051910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.064366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4dac8-5d1d-476c-b650-0db7e55822ec None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.119363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.120334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.120668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.120901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.122284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.123884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159b0c3-6d65-4182-97c5-2841a0546563 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.126016 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 138/269] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:29 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:30.127080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea1bd-a53b-4d47-97f2-f8267bbc47a2 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.127375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea1bd-a53b-4d47-97f2-f8267bbc47a2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.127661 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea1bd-a53b-4d47-97f2-f8267bbc47a2 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.129232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 139/270] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:30.130122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea1bd-a53b-4d47-97f2-f8267bbc47a2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.177103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e0d2a-e40c-432c-9bd5-9bf778b8684b None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.222039 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.228977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.230726 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.231246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.231738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.232823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=5144e4689f8246b0b8f38f83f0541692) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:30.233348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.235068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.242357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 132/271] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/role_assignments?user.id=5144e4689f8246b0b8f38f83f0541692&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=519d740b9c3d46f3902513905864fa5d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:30.243614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.253380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea1bd-a53b-4d47-97f2-f8267bbc47a2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.284857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.305221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.306401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.306694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.308280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.335431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-78dfffdf-dbf5-4557-8c71-3441a9564dd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:30.348797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:30.349889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:30.351179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 140/272] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:30.351917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.351917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.352052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.400962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:30.402732 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:30.456652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f862b47-a656-4297-91e5-123d25581a93 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.487389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 141/273] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:30.488301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.502756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.503841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.504338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.504852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.505907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=5144e4689f8246b0b8f38f83f0541692) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:30.506449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.508118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.516180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 133/274] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/role_assignments?user.id=5144e4689f8246b0b8f38f83f0541692&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:30.539393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.621189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.677561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b4244-d620-47f8-bfe9-07ba266de09a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.704035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf9c56-8338-4abc-848a-6b06e9044859 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.726774 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.727762 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.732432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.732432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.732432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.732432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.734390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 134/275] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:30.739903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.744858 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.789268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.790109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.790411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.790702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.791546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:30.791874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.793812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd7450-b0c2-455b-9f61-f3dcb8eda677 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.804965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 142/276] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:30.806596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbec39-d4ca-4c75-a3ab-9aa124f52422 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.806899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbec39-d4ca-4c75-a3ab-9aa124f52422 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.807299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbec39-d4ca-4c75-a3ab-9aa124f52422 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.809028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 143/277] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:30.814761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbec39-d4ca-4c75-a3ab-9aa124f52422 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:30.865826 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cbe679-19a4-48e4-ab55-6e559523a678 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.894118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:30.904272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.905003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.905290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.905570 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.906306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:30.907775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e3176-4665-4ed8-a9a5-2f82424375f4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:30.928095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 135/278] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:30 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:10:30.929608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f392-86d0-467e-8657-64fd88c5abbc None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.930061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f392-86d0-467e-8657-64fd88c5abbc None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.930526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720f392-86d0-467e-8657-64fd88c5abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.939012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbec39-d4ca-4c75-a3ab-9aa124f52422 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:30.950534 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-9720f392-86d0-467e-8657-64fd88c5abbc None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:30.951866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-9720f392-86d0-467e-8657-64fd88c5abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:30.981545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:30.982450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:30.982737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:30.983022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] PATH_INFO: `/v3/users/2d694e243ed4408795fd83b4de225485` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:30.999449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] RBAC: Authorizing `identity:get_user(user_id=2d694e243ed4408795fd83b4de225485)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:31.001126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d03f3f-e16f-4cbe-b75d-314c9da41e93 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:31.003274 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 144/279] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:30 2022] GET /identity/v3/users/2d694e243ed4408795fd83b4de225485 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:31.004446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5c54-7aa7-49c1-8dc0-2d38daa81a2e None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.004761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5c54-7aa7-49c1-8dc0-2d38daa81a2e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.005047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea5c54-7aa7-49c1-8dc0-2d38daa81a2e None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.010500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 145/280] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:31.010500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea5c54-7aa7-49c1-8dc0-2d38daa81a2e None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.042924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 136/281] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:31.044600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.044914 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.045194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.065450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:31.065667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.066708 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:31.149012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea5c54-7aa7-49c1-8dc0-2d38daa81a2e None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.150146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 137/282] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:31.151067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.200741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.203964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.203964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.203964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.203964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, user.id=2d694e243ed4408795fd83b4de225485, scope.system=all) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:31.204353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:31.205784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:31.211991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 146/283] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/role_assignments?user.id=2d694e243ed4408795fd83b4de225485&scope.system=all&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:31.212956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.225980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.310574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd8309-8fc9-44d7-adf6-4ced57dd6b41 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.366014 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.375684 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27499c3-4afb-424d-92d1-331d3a8e965d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.379677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.380883 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.381366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.381884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.399379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:31.412042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:31.413197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:31.414068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:31.417126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 138/284] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:31.426106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.430225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.431879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.431879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.431879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.445879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:31.454431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:31.455468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:31.456348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 147/285] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:31.462059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.594243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281ec93-333e-413f-b6ea-bfc3ffd533a5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.596712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.614706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f9a86-49d9-4609-9e6f-f521b9ec8feb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.636604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.637339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.637693 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.638070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.638942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:31.639472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:31.641098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:31.653140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 139/286] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:31.662500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.674585 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.675871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.676498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.677276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.678919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:31.684883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:31.686944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:31.698897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 148/287] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:31.709605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.756894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.796021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a777c-e8b7-41b3-aa97-d46c36932efb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.834617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:31.836150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:31.836525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:31.836821 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] PATH_INFO: `/v3/users/713d1c6f09974e31b89121156e431c1f` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:31.838434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] RBAC: Authorizing `identity:get_user(user_id=713d1c6f09974e31b89121156e431c1f)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:31.841048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:31.843185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 140/288] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/users/713d1c6f09974e31b89121156e431c1f => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:31.849145 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:31.895532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:31.976643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32eb22-8344-49ce-a23f-e35d759737d4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:31.991493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08aad1-6380-433c-abc1-b5a50390b7b8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:32.017655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.018490 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.018783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.019062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.093902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.094821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.095114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.095439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] PATH_INFO: `/v3/users/4b5af7af380d45c083e789876db4f1b2` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.096680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] RBAC: Authorizing `identity:get_user(user_id=4b5af7af380d45c083e789876db4f1b2)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.098246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.100371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 149/289] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/users/4b5af7af380d45c083e789876db4f1b2 => generated 297 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:32.101513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.103089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.106406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:32.144324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:32.227426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 141/290] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:32.233844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:32.367714 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:32.376536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a40727-4d0c-4436-9073-46d528695734 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:32.413856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.414687 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86122358-4ded-408b-8522-86c7362a4f95 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:32.414913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.415277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.415633 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.416964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.418671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.420500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 150/291] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:32 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:32.426129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:32.520961 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.521892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.522248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.522602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.523900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.525400 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.532636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 142/292] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:32 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:32.535482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50a772-24c7-49e6-9744-7c72c34367f2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:32.543466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:32.547042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:32.577004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.577916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.578205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.578491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/4b5af7af380d45c083e789876db4f1b2/roles/c0f70d47fd324f5ba11fc34d35c801b5` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.580895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, user_id=4b5af7af380d45c083e789876db4f1b2, role_id=c0f70d47fd324f5ba11fc34d35c801b5)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.582396 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65fa73d-3074-4f7f-b722-a3eade15db8a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.611944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 151/293] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:10:32 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/users/4b5af7af380d45c083e789876db4f1b2/roles/c0f70d47fd324f5ba11fc34d35c801b5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:10:32.701349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9076-700f-4717-ba95-a1931cd02f05 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:32.747188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:32.747879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:32.748170 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:32.748479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:32.749353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=713d1c6f09974e31b89121156e431c1f) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:32.749673 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:32.751177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6ffb0-1702-407b-8f0d-b0a48d446a7d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:32.758825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 143/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:32 2022] GET /identity/v3/role_assignments?user.id=713d1c6f09974e31b89121156e431c1f&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:33.849192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecb50-f00d-4856-b0d2-7e81e1027c9a None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.849845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecb50-f00d-4856-b0d2-7e81e1027c9a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.849845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389ecb50-f00d-4856-b0d2-7e81e1027c9a None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.851411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 152/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:33.858767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddcf2-5913-4446-9efc-7a21fc343588 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.859845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddcf2-5913-4446-9efc-7a21fc343588 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.860355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cddcf2-5913-4446-9efc-7a21fc343588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.866713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbaae-ceb1-4683-9c5e-84772ce198c9 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.867382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbaae-ceb1-4683-9c5e-84772ce198c9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.867881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cbaae-ceb1-4683-9c5e-84772ce198c9 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.869804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 153/296] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:33.878183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.878470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.878761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.881265 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d4cddcf2-5913-4446-9efc-7a21fc343588 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:33.882469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d4cddcf2-5913-4446-9efc-7a21fc343588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:33.901127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:33.903475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:33.921406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:33.965952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:33.974384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 144/297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:33.982551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2aa74-71fe-4edb-ad46-0098403ea535 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.983143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2aa74-71fe-4edb-ad46-0098403ea535 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.983591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2aa74-71fe-4edb-ad46-0098403ea535 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.986331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 145/298] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:33.992000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:33.992510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:33.993283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:33.998384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 154/299] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:33.999589 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.014496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:34.016453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:34.096649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 146/300] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:34.097947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.118342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0f280-2509-43f3-943b-397bd5e11af2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.159054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.160360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.160956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.161705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.174532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:34.185022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:34.188454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 155/301] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:34.189524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.200314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.221612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d680bb-ee52-4691-af04-5bb30ad9991a None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.254858 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.267753 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.268596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.268883 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.269159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.283242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:34.291985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:34.294244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 147/302] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:34.295183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.319000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880cd0-8860-4859-a179-21ac2741d15a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.370017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.370954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.371246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.371537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.372779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.374254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.376063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 156/303] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:34.376705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.423118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e8c24-37e3-4ff6-aee1-a7a4530c38ff None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.462041 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.465509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.466252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.466543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.466859 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.467968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:34.468327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.469720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.482988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 148/304] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:34.482988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.495637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838de84-6961-491a-a6f7-596d70fef1e9 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.531984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.532695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.532973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.533255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.534027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:34.534334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.535723 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.535865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.545617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 157/305] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:34.546659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.600007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e613c8d-c1cf-4661-978c-423d624526f7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.647686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.648380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.648702 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.648987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] PATH_INFO: `/v3/groups` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.649731 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.655577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.670814 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950b299-db9b-43ea-88be-b1406a003694 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.671238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 149/306] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 20:10:34 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:34.672142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.726657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.727987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.728503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.728960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] PATH_INFO: `/v3/users/a15125f0e97d488092c320050a7ca61b` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.737705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.759329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] RBAC: Authorizing `identity:get_user(user_id=a15125f0e97d488092c320050a7ca61b)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.761768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.764337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 158/307] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/users/a15125f0e97d488092c320050a7ca61b => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:34.770460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.825489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6648e-6416-4080-82df-007c77d5a83d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.836446 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:34.879953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.881759 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.882341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.882873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] PATH_INFO: `/v3/users/2d694e243ed4408795fd83b4de225485` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.884422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] RBAC: Authorizing `identity:get_user(user_id=2d694e243ed4408795fd83b4de225485)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.886653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.888372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 150/308] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/users/2d694e243ed4408795fd83b4de225485 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:34.892419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a383e4-70d5-4a3c-ad8c-734eabcffc7a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:34.894094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:34.940194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:34.940811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:34.950065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:34.952275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:34.952691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, user.id=a15125f0e97d488092c320050a7ca61b, scope.system=all) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:34.952939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:34.956387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a3beb-1a46-4f99-9d04-da4b29118ae1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:34.967311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 159/309] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:10:34 2022] GET /identity/v3/role_assignments?user.id=a15125f0e97d488092c320050a7ca61b&scope.system=all&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:35.036847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9645fb8-1146-4994-bf74-c7249d7b5430 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:35.082034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:35.082930 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.083219 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.083504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] PATH_INFO: `/v3/system/users/2d694e243ed4408795fd83b4de225485/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.085173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d694e243ed4408795fd83b4de225485, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:35.086619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf265c3-94e8-4f19-ae96-61a6d829d905 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:35.098339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 151/310] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 20:10:34 2022] PUT /identity/v3/system/users/2d694e243ed4408795fd83b4de225485/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:10:35.246645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e146f-d90a-468c-b03a-a27f2c5e0f11 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.247022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e146f-d90a-468c-b03a-a27f2c5e0f11 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.247206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e146f-d90a-468c-b03a-a27f2c5e0f11 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.249025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 160/311] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:35.257535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.258007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.258397 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.278494 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:35.279757 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:35.320392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:35.358458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 152/312] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:35.382324 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e146f-d90a-468c-b03a-a27f2c5e0f11 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:35.450793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:35.505687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e146f-d90a-468c-b03a-a27f2c5e0f11 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:35.547222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:35.548047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.548335 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.548663 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.561201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:35.569106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:35.570094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:35.570970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 161/313] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:35.576923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:35.687966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b48f36-d129-4371-856a-d386602c95d0 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:35.730816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:35.731500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.731783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.732061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.732857 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] RBAC: Adding query filter params (name=reader) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:35.733166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:35.734573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:35.744940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 153/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:35.750750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:35.861457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de31918-27e3-4cd6-9a94-560b6e8b0ce0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:35.880145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:35.905221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:35.906166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:35.906510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:35.906797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] PATH_INFO: `/v3/users/4b5af7af380d45c083e789876db4f1b2` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:35.908041 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] RBAC: Authorizing `identity:get_user(user_id=4b5af7af380d45c083e789876db4f1b2)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:35.909562 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:35.911576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 162/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:35 2022] GET /identity/v3/users/4b5af7af380d45c083e789876db4f1b2 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:35.917806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:35.957478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:36.036645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ac436-92d2-40de-9250-222a1b074fca None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:36.076073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:36.077267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:36.077781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:36.078289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:36.174708 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:36.176234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:36.221941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:36.298111 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 154/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:10:36.303801 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:36.417614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d5204-3a8b-433e-9730-601e36b17b7d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:36.454828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:36.455714 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:36.456002 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:36.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:36.457553 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:36.459029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb9ca3-b45c-44f4-857d-da8b396d3345 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:36.461124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 163/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:36 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:36.467271 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:36.578271 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172d8f5-a623-4d85-87a4-005573e8eb46 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:36.610010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:36.616809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:36.617479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:36.617767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:36.618045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:36.618895 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786, user.id=4b5af7af380d45c083e789876db4f1b2) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:36.619218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:36.621450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c089f-1be2-49be-9b14-688fe1cae4be None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:36.629264 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 155/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:10:36 2022] GET /identity/v3/role_assignments?user.id=4b5af7af380d45c083e789876db4f1b2&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:37.222469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0849-dba5-491e-a982-c98f69cff39c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.222998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0849-dba5-491e-a982-c98f69cff39c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.223091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd0849-dba5-491e-a982-c98f69cff39c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.224602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 164/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:37.229967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576eb5-bae8-4f9c-9198-28fdccc0ad94 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.230560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576eb5-bae8-4f9c-9198-28fdccc0ad94 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.230560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576eb5-bae8-4f9c-9198-28fdccc0ad94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.248027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-84576eb5-bae8-4f9c-9198-28fdccc0ad94 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:37.249271 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-84576eb5-bae8-4f9c-9198-28fdccc0ad94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:37.325697 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 156/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:37.348863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd0849-dba5-491e-a982-c98f69cff39c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.360163 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ac19b-e14f-4eb4-86a5-a65546612db1 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.360480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ac19b-e14f-4eb4-86a5-a65546612db1 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.360770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045ac19b-e14f-4eb4-86a5-a65546612db1 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.362347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 157/321] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:37.367722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.368023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.368312 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.387125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:37.388369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:37.416785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:37.471582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd0849-dba5-491e-a982-c98f69cff39c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:37.472211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 158/322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:37.494820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.512370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:37.513556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.514014 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.514470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.516106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:37.517840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c1ce1-21da-452f-9b04-bbb75d93de64 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:37.519586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 165/323] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:37.521790 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb58d2-4a8c-4534-b33f-be4653f8548b None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.522085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb58d2-4a8c-4534-b33f-be4653f8548b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.522370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb58d2-4a8c-4534-b33f-be4653f8548b None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.524251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 166/324] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:37.529157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb58d2-4a8c-4534-b33f-be4653f8548b None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.565805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:37.614498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c13e78-84f5-443f-8464-e7859623c3d2 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:37.638447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb58d2-4a8c-4534-b33f-be4653f8548b None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:37.660474 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:37.661874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.662033 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:37.662393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.663030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.678639 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:37.679328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.679621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.679906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] PATH_INFO: `/v3/groups` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.680652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] RBAC: Authorizing `identity:create_group()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:37.682387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42b1d-1bb3-4b58-a80f-722b80d9d0ed None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:37.695438 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:37.710357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 167/325] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 20:10:37 2022] POST /identity/v3/groups => generated 226 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:37.710357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.710357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.710357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:37.710917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:37.711839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:37.712812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 159/326] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:37.724574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.734643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:37.735902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:37.817653 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 168/327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:37.842246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.851629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d846161a-ec43-4738-ba48-5fb38d712bf5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:37.872225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:37.893404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:37.894493 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:37.894993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:37.895482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:37.896375 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:37.896792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:37.898299 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:37.912438 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 160/328] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:37.913524 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:37.970216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050128f2-73ad-452f-8836-143be46cfce3 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:37.987911 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:38.019344 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.020565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.021026 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.021467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.037755 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:38.046342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:38.047394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:38.048297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:38.050476 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 169/329] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:38.056456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:38.070197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe04b2d-31ca-4ce7-84fe-c329db9d11c7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:38.120735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.122051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.122289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.122577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] PATH_INFO: `/v3/users/a15125f0e97d488092c320050a7ca61b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.123858 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] RBAC: Authorizing `identity:get_user(user_id=a15125f0e97d488092c320050a7ca61b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:38.127884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:38.128323 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 161/330] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:37 2022] GET /identity/v3/users/a15125f0e97d488092c320050a7ca61b => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:38.134467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:38.180415 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fd65a-5cd9-41da-b3af-926557251f2a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:38.195474 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:38.231170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.234295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.234295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.234295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.234916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:38.235252 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:38.237133 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:38.252023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 170/331] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:38.262306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:38.268838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d8bfa-b6b4-4da2-8036-39c6b85e4a25 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:38.312070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:38.317543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.318425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.318713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.318993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] PATH_INFO: `/v3/system/users/a15125f0e97d488092c320050a7ca61b/roles/c0f70d47fd324f5ba11fc34d35c801b5` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.320573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a15125f0e97d488092c320050a7ca61b, role_id=c0f70d47fd324f5ba11fc34d35c801b5)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:38.322082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:38.333531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 162/332] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 20:10:38 2022] PUT /identity/v3/system/users/a15125f0e97d488092c320050a7ca61b/roles/c0f70d47fd324f5ba11fc34d35c801b5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:38.413370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3e84-9eb2-434a-9773-2a9e551abc33 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:38.458901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.459800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.460086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.460364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] PATH_INFO: `/v3/users/2d694e243ed4408795fd83b4de225485` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.461702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] RBAC: Authorizing `identity:get_user(user_id=2d694e243ed4408795fd83b4de225485)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:38.463177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696bd0a-4c77-4137-a057-d09512993d6e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:38.465781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 171/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:38 2022] GET /identity/v3/users/2d694e243ed4408795fd83b4de225485 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:38.473224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:38.623170 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134bcf5-ee46-4ca1-aaaf-eb2674b9de11 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:38.634166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:38.665755 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:38.666484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:38.666770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:38.667228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:38.668093 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] RBAC: Adding query filter params (role.id=dc20953902b3457186b2eb9d4ec05d6b, user.id=2d694e243ed4408795fd83b4de225485, scope.system=all) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:38.668424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:38.669950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa11bac-325e-408a-90db-2ee6a964ea83 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:38.684149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 163/334] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:10:38 2022] GET /identity/v3/role_assignments?user.id=2d694e243ed4408795fd83b4de225485&scope.system=all&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 510 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:40.551050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f568-8a19-4e02-bfc8-fc8c3c9386e4 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:40.551554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f568-8a19-4e02-bfc8-fc8c3c9386e4 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:40.551554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f568-8a19-4e02-bfc8-fc8c3c9386e4 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:40.553266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 172/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:40.559557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:40.560039 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:40.560141 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:40.578602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:40.579900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:40.674709 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 164/336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:40.711730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f568-8a19-4e02-bfc8-fc8c3c9386e4 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:40.840819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f568-8a19-4e02-bfc8-fc8c3c9386e4 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:40.842928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:40.880602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:40.881421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:40.881709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:40.881987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:40.895620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:40.904447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:40.905521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:40.906489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 173/337] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:40.912125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:41.040254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:41.045104 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b67d8-7455-4c1a-80cc-b958547d235d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:41.091027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:41.091629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.091912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.092187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.092987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:41.093295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:41.094680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0845556-76f4-4254-8a8d-1bf1850fe13f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:41.109041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 165/338] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:41.114907 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:41.225783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434871b5-3343-4cbc-8f26-e0a88578dffb None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.226123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434871b5-3343-4cbc-8f26-e0a88578dffb None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.226351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434871b5-3343-4cbc-8f26-e0a88578dffb None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.227990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 166/339] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:41.234433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.234741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.234995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.262111 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:41.263332 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:41.302892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3a019-f094-4728-bc6c-ce9a62e3b310 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:41.366954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:41.368101 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.368630 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.369103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.393389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 167/340] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:41.420431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:41.426295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:41.527240 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:41.528729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b5ffb-99af-4bd4-bded-5cb20eb751ca None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:41.576329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:41.591664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5619a6-37b2-4f5b-87fc-6a9a4fea8903 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:41.656653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:41.657506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.657802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.658084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.669494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 174/341] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:41.671424 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ca99-f833-41d5-8eb4-2d9ee376b7ae None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.671864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ca99-f833-41d5-8eb4-2d9ee376b7ae None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.672372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ca99-f833-41d5-8eb4-2d9ee376b7ae None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.674100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 175/342] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:41.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1ca99-f833-41d5-8eb4-2d9ee376b7ae None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:41.681104 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0e8e8afe-ee58-4712-bbc3-d3b3524ad44d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:41.693525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:41.694898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:41.695828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 168/343] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:41.696682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.696766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.697024 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.724282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:41.725546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:41.786917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:41.855961 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 169/344] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:41.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:41.913207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1ca99-f833-41d5-8eb4-2d9ee376b7ae None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:41.990234 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:41.991142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:41.991433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:41.991719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:41.993404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:41.995065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:41.996892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.005002 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 176/345] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:41 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:42.006495 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.021531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa5f27-510d-4a0f-9e21-ac7965f78ba8 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.067491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.068456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.069037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.069468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.070651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:42.071116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:42.072822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:42.085070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 170/346] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:42.086223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.212430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824157a-1c42-4630-ae66-88a62bfe1a6f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.231317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4fe06-6113-4624-adec-9ce1a969cdc5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.251025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.268301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.269273 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.269568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.269852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.274618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.275510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.275803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.276081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.283652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:42.283862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:42.285206 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:42.287273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 171/347] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:42.288065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-71adffc7-9f26-42db-9183-31418a39beb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:42.298925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:42.299989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:42.300993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:42.304441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.304788 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 177/348] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:42.306596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e027c-cd59-4670-90c4-aaa1e3604474 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.306876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e027c-cd59-4670-90c4-aaa1e3604474 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.307251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e027c-cd59-4670-90c4-aaa1e3604474 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.310086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 178/349] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:42.312112 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e027c-cd59-4670-90c4-aaa1e3604474 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.422814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868eaf1-69e4-4c7e-a4b0-fdc38feaceeb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.448577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e027c-cd59-4670-90c4-aaa1e3604474 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.463904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.465080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.465589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.466138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.491277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.492094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.492861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.492861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.494231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=18c9cf67e140413ea746a42df18a6f98) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:42.494231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:42.495572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cceafc-969f-4d99-af30-7e85451c0129 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:42.502155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 179/350] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=519d740b9c3d46f3902513905864fa5d => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:42.503266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.503569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.504796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.535566 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:42.536852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:42.581762 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:42.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31c7d9-3b53-436b-9443-60c4c4e6fc11 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:42.601467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.629259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.655096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 180/351] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:42.656020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.738433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 172/352] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:42.740059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd0d8a-372f-4074-ad6d-b11219df3fbb None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.740360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd0d8a-372f-4074-ad6d-b11219df3fbb None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.743466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd0d8a-372f-4074-ad6d-b11219df3fbb None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.746169 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 173/353] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:42.748256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd0d8a-372f-4074-ad6d-b11219df3fbb None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.830672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357641ba-a1ee-49c4-9c34-23a459580e35 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.873196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.873940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.874235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.874699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.875533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:42.875854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:42.878086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:42.889136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 181/354] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:42.890093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:42.900782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd0d8a-372f-4074-ad6d-b11219df3fbb None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:42.909852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.944487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:42.949749 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:42.950879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.952246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.952797 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:42.965543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-980acf90-b812-4c4d-9a7f-bca66ebc9ac0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:42.974183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:42.975201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:10:42.976089 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:42.978766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 174/355] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:42.982381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:42.983010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:42.983493 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.005593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:43.007131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:43.007668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de836d-9c34-4a28-9f0f-21d372fbfb34 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.055660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.055660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.055660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.055660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.057163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.058802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.061935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 182/356] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:42 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.063170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.110654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 175/357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:43.111619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.249823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab510058-d754-4a32-9113-5be25a40f377 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.257612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.265534 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca48ac-2859-4ef4-9ec8-f8c51133a069 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.273302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.307547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.308294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.308652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.308933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.311519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.312272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.312660 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.313007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.313872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] RBAC: Adding query filter params (name=reader) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:43.314188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.315594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.325503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 176/358] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.326348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.398260 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.400637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.501324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578b480-9961-4797-8f54-e737dbc5ebec None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.545472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.546266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.546584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.546866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.548189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.549651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.553467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 183/359] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:43.554382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.565818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 177/360] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.567037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.576171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.603129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.694000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab875dd3-7ccf-4303-8a23-a927d5d34b1c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.707360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f89755-8247-4775-b421-e7baf66318d5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.741288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.742133 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.742424 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.742709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.754678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.755563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.755843 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.756119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] PATH_INFO: `/v3/users/a15125f0e97d488092c320050a7ca61b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.759356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a15125f0e97d488092c320050a7ca61b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.762163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:43.762317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721fd5-a7ba-4f7f-afb6-c11b096760f1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.763597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 178/361] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/users/a15125f0e97d488092c320050a7ca61b => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.765758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301554e9-9245-430c-b5dd-1172b66b93bd None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.766052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301554e9-9245-430c-b5dd-1172b66b93bd None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.766330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301554e9-9245-430c-b5dd-1172b66b93bd None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.767943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 179/362] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:43.768773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301554e9-9245-430c-b5dd-1172b66b93bd None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:43.778446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:43.779637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:43.780542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 184/363] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:43.781293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.887267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.902830 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:43.916068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301554e9-9245-430c-b5dd-1172b66b93bd None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.930514 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8dd9-0542-4d1f-a11e-9fdca431d181 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:43.969231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.970280 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.970712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.971006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.972289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.973793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55523222-64c3-4647-ade1-79d5cef9e4ae None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.975602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 180/364] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.977340 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:43.978126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.978427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.978766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.979031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:43.980216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:43.980935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:43.982082 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=18c9cf67e140413ea746a42df18a6f98) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:43.982482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:43.984012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:43.992986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 185/365] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:43.993664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:43.999905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:44.001209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:44.103346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 181/366] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:44.105239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.145634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5262e-1d43-4b30-95d2-d46e391a8838 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.203972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.204915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.205282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.205735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.206677 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] RBAC: Adding query filter params (role.id=c0f70d47fd324f5ba11fc34d35c801b5, user.id=a15125f0e97d488092c320050a7ca61b, scope.system=all) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:44.207010 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:44.209735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:44.216715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.231092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 186/367] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:10:43 2022] GET /identity/v3/role_assignments?user.id=a15125f0e97d488092c320050a7ca61b&scope.system=all&role.id=c0f70d47fd324f5ba11fc34d35c801b5 => generated 510 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:44.232285 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.249705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.263591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e2c9c-0438-41c5-9468-e345fbff7d53 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.319719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.320459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.320941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.321230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.322421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:44.322743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:44.325310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:44.337632 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 182/368] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:44.340019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.378432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2815e-e474-4d0c-918f-221c7a9b6e8a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.422013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.422806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.423094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.423373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.425243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:44.426838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:44.429012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 187/369] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:44.430087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.488528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca090e59-6064-4505-ae9e-23cc37effac9 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.533169 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.533993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.534285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.534571 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.548863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:44.558677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:44.561806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 183/370] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:44.563867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.582891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.609468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.633374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91185-c60a-4795-b14e-0766c65364be None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.681492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.682171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.683213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.683738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.684157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e0178-99dc-40aa-9a5a-a30715e18b64 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:44.721749 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:44.722434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:44.722725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:44.723011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:44.724015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:44.724192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:44.725694 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:44.737962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 184/371] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:44.739125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.807301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:44.812377 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:44.893368 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.912122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:44.945646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 188/372] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:44.951816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:44.954956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d9f6d-5b5f-46cd-b805-c6e825d3b951 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.053380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.054408 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.055610 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.056085 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.058174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:45.060254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.062736 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.082695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 185/373] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:45.087488 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:45.108252 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e44fb0-121f-487d-b664-04e5137c3488 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.161129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.164753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.165094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.165389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.167156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.171236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.171884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 189/374] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:44 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:45.178167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:45.240819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:45.314176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83145f6a-89e1-4cac-8858-80f764f10282 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.359510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.360190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.360508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.360788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.373517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:45.379285 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d098e-2489-42f0-9d15-d510dfa03463 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.432686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.433728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.434115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.434469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.435845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.440114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af8c62-2a21-453f-b731-bcde75596673 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.442445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 190/375] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:45.443828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a547d0-d0b6-4f84-8be0-00783c997f6d None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.448796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a547d0-d0b6-4f84-8be0-00783c997f6d None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.449405 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a547d0-d0b6-4f84-8be0-00783c997f6d None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.455114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 191/376] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:45.455114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a547d0-d0b6-4f84-8be0-00783c997f6d None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:45.475121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.476646 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563d65-eb6c-4a27-b07e-9bed24e7c24f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.613518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:45.615463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 186/377] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:10:45.616995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.617287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.617564 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.621222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a547d0-d0b6-4f84-8be0-00783c997f6d None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.637189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:45.638383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:45.674353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.675433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.675962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.676506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.677717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:45.678181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:45.679422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.681576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.691240 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 192/378] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=519d740b9c3d46f3902513905864fa5d => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:45.692859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:45.724574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 187/379] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:45.752431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:45.933682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8655b6-2873-4f84-9866-99a6fc56937e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.977472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9f007-1a7d-4ae3-b1fc-63fc2a3e86f6 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:45.983087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:45.983981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:45.984265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:45.984568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] PATH_INFO: `/v3/projects/0064d8382e454dc1bf5476c00fe5a47a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:45.986145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] RBAC: Authorizing `identity:get_project(project_id=0064d8382e454dc1bf5476c00fe5a47a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:45.987608 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:45.989844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 193/380] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/projects/0064d8382e454dc1bf5476c00fe5a47a => generated 344 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:45.995774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.053037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.053863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.054153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.054537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.062681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:46.080755 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:46.093880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:46.094948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:46.096281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 188/381] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:46.103181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.148328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f16ad-dcc0-4e73-8fe8-eb1a23db7204 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.157262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:46.202904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.203903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.204192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.204493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] PATH_INFO: `/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.213074 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] RBAC: Authorizing `identity:get_group(group_id=1dd049a94e1a4bb1b0b11a7153b9397b)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:46.215758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:46.217989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 194/382] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:45 2022] GET /identity/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:46.223965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.257073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e094d-f95a-44eb-b9f3-4b5cc6338e70 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.324130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.324859 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.325150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.325431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.326371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:46.326761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:46.328248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:46.341252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 189/383] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:46.347103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.416910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:46.451764 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9340-47dc-422b-97bc-44687e8cc932 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.536798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.537420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.537713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.537997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.538865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] RBAC: Adding query filter params (group.id=1dd049a94e1a4bb1b0b11a7153b9397b, role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=0064d8382e454dc1bf5476c00fe5a47a) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:46.539186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:46.544091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:46.558110 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 195/384] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:46 2022] GET /identity/v3/role_assignments?group.id=1dd049a94e1a4bb1b0b11a7153b9397b&scope.project.id=0064d8382e454dc1bf5476c00fe5a47a&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 260 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:46.567129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:46.578840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd27a1b-3c5b-4753-8b23-c11b3d939962 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.622866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.623758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.624047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.624330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.626011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:46.627597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:46.629987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 190/385] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:46 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:46.636167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.776174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922f257-b69c-4f34-8714-0ed675ea5b5a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.831033 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:46.831902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.832227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.832537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.833800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:46.835291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5c7c7-d3b2-49a9-ba4c-48c606252708 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:46.837277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 196/386] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:46 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:46.844132 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:46.882852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ecae-3e88-4f71-939c-121d82223e47 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.883186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ecae-3e88-4f71-939c-121d82223e47 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.883414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150ecae-3e88-4f71-939c-121d82223e47 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.885023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 197/387] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:46.893458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:46.893458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:46.893458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:46.933486 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:46.935036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:46.975186 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3897a-f020-4e6f-89a5-d9dea0bcb87e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:46.999544 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:47.025167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:47.026108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:47.026395 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:47.026678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/groups/b6d02a35ddb042ff89015301bbf3541a/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:47.035675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, group_id=b6d02a35ddb042ff89015301bbf3541a, role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:47.037263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:47.068557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 191/388] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:10:46 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/groups/b6d02a35ddb042ff89015301bbf3541a/roles/519d740b9c3d46f3902513905864fa5d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:47.084826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 198/389] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:47.113277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:47.139085 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:47.234963 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091e494-d421-4a11-82dd-a7443c355576 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:47.274631 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:47.275467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:47.275756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:47.276040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:47.288824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:47.297116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:47.298291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:47.299231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 192/390] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:47.304844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:47.429175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829361b9-6936-40cb-af20-f6c8392fe6f6 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:47.469870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:47.470592 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:47.470883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:47.471167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:47.471964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:47.472277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:47.473775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:47.485787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 199/391] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:47.489459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:47.583445 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:47.603662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d807-c440-42e1-adbe-c65f021ed130 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:47.645661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:47.646546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:47.646830 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:47.647114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:47.648354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:47.649856 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:47.652614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 193/392] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:47 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:47.661557 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:47.688885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:47.791219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b00f5-fcd1-4bc9-8316-7fe97692b38d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:47.838646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:47.839780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:47.840071 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:47.840418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:47.841681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:47.843150 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75947a1f-b6dd-4cb9-9da6-d589e6acdd9a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:47.845066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 200/393] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:47 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:47.850862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:47.989247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3bbb6-9372-4da5-bd22-651442660046 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:48.024590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c310-8017-4395-9402-1ffce924d7d6 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.025158 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c310-8017-4395-9402-1ffce924d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.025158 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69c310-8017-4395-9402-1ffce924d7d6 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.029098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 201/394] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:48.043480 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.043817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.044059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.050170 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:48.051073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.051360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.051641 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98/groups/b6d02a35ddb042ff89015301bbf3541a/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.053718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] RBAC: Authorizing `identity:create_grant(project_id=18c9cf67e140413ea746a42df18a6f98, group_id=b6d02a35ddb042ff89015301bbf3541a, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:48.055246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:48.072079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:48.073469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:48.073884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 194/395] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:10:47 2022] PUT /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98/groups/b6d02a35ddb042ff89015301bbf3541a/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:48.183111 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 202/396] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:48.231814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:48.255228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:48.395983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3b32-c17d-41f9-81a1-75a78f072323 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:48.485162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:48.486513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.487025 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.487519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.521061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:48.536181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:48.537227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:48.538121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:48.542730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 195/397] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:48.553552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:48.588820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:48.707457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc38c4a-8fac-487e-9ebe-bdad97da59c0 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:48.750034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:48.751146 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.751676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.752174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.753201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] RBAC: Adding query filter params (name=member) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:48.753750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:48.755353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0945f2c-3e9a-44f7-bab6-d88ba01838ee None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:48.764699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 203/398] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:48.770775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:48.878180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:48.896453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a52c-ec54-457a-bf2c-1f8d22feaf2e None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.897121 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a52c-ec54-457a-bf2c-1f8d22feaf2e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.897618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a52c-ec54-457a-bf2c-1f8d22feaf2e None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.899232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 204/399] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:48.905005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500495a-c971-4e51-aa12-30982b37495a None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:48.905297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500495a-c971-4e51-aa12-30982b37495a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:48.905583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500495a-c971-4e51-aa12-30982b37495a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:48.940900 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-2500495a-c971-4e51-aa12-30982b37495a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:48.944865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-2500495a-c971-4e51-aa12-30982b37495a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:48.952477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b8302-250e-4278-ac62-4e655b5e6158 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:49.035687 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:49.037010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.041008 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.041567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.045306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:49.047182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:49.051883 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 196/400] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:48 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:49.058239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:49.098607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 205/401] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:49.136797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500495a-c971-4e51-aa12-30982b37495a None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:49.156771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:49.287092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500495a-c971-4e51-aa12-30982b37495a None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:49.325341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41755193-e032-47ee-af30-7cdcb58c803b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:49.340498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:49.341356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.341647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.341927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.363338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:49.372323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:49.381521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:49.382401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:49.383070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:49.383070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.383070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.383070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.383370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:49.383455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:49.385550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 197/402] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:49 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:49.386099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 206/403] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:49.392775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:49.394078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:49.412862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:49.550715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:49.609579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92978665-7cde-4c30-b151-8e9935c232bd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:49.631884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498bdd-1d72-4088-9c5a-8a696b2c3392 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:49.661255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:49.661844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.662131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.662413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.663169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:49.663485 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:49.664928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:49.684991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 207/404] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:49.693679 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:49.700017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:49.701628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.702198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.702701 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/groups/b6d02a35ddb042ff89015301bbf3541a/roles/dc20953902b3457186b2eb9d4ec05d6b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.708306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, group_id=b6d02a35ddb042ff89015301bbf3541a, role_id=dc20953902b3457186b2eb9d4ec05d6b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:49.711086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1d904-d36c-4693-a3f6-6e3080bd4916 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:49.749245 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 198/405] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:10:49 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/groups/b6d02a35ddb042ff89015301bbf3541a/roles/dc20953902b3457186b2eb9d4ec05d6b => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:49.759958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2867b-03e1-43a7-81c5-6d18c49fe7da None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.761552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2867b-03e1-43a7-81c5-6d18c49fe7da None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.762309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2867b-03e1-43a7-81c5-6d18c49fe7da None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.765566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 199/406] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:49.774153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:49.775079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:49.775871 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:49.841255 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:49.843356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:49.987715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47c15-63cd-4f67-8763-c7e6287c64ab None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.002431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.003151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 200/407] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:50.032219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.033559 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.047715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.048626 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.048917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.049198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.053249 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:50.054735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:50.056901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 208/408] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:49 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:50.063465 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.186431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b6fb-1099-45f1-87af-4e0eaf7311c0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.239073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.240474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.241084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.241684 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.243303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:50.245191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:50.248036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 209/409] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:50 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:50.254612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.270091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c120a-d338-4a54-ba88-ce3177f5924f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.335199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.336259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.337649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.338116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.339762 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.359510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-4fad3e6b-9154-4c2e-a73b-3278e1e264f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:50.375538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:50.376650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:50.377643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:50.381426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436db810-dfd0-4163-91c7-d2f3cd4c71a8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.385356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 201/410] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:50.388379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cacf9-bb4f-45cc-a581-9465f469d2cf None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.389081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cacf9-bb4f-45cc-a581-9465f469d2cf None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.389624 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cacf9-bb4f-45cc-a581-9465f469d2cf None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.394181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 202/411] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:50.395845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cacf9-bb4f-45cc-a581-9465f469d2cf None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.421912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.423134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.423724 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.424178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/groups/b6d02a35ddb042ff89015301bbf3541a/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.426975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] RBAC: Authorizing `identity:create_grant(project_id=f707fa233ec64e4a8da5e1ce11fd2786, group_id=b6d02a35ddb042ff89015301bbf3541a, role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:50.428673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702df41-468a-4c47-911c-99d7b6ce7b78 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:50.445984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 210/412] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:10:50 2022] PUT /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786/groups/b6d02a35ddb042ff89015301bbf3541a/roles/519d740b9c3d46f3902513905864fa5d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:50.453443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba931-6159-492f-9630-30fb620410a8 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.454037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba931-6159-492f-9630-30fb620410a8 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.454534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba931-6159-492f-9630-30fb620410a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.482815 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-682ba931-6159-492f-9630-30fb620410a8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:50.484546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-682ba931-6159-492f-9630-30fb620410a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:50.512182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.601971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cacf9-bb4f-45cc-a581-9465f469d2cf None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.670579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.671695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.672153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.672650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.673977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:50.674506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:50.677061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:50.700561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 203/413] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:50.705417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.710482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 211/414] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:50.736439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba931-6159-492f-9630-30fb620410a8 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:50.859128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.900327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d1b40-4ab9-4314-abae-fb3aacb2495c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.948799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:50.968050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba931-6159-492f-9630-30fb620410a8 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:50.992595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:50.993489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:50.993985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:50.994269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] PATH_INFO: `/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:50.996977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] RBAC: Authorizing `identity:get_group(group_id=1dd049a94e1a4bb1b0b11a7153b9397b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:50.998469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:51.012066 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 204/415] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:50 2022] GET /identity/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b => generated 229 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:51.014191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:51.053560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.054398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.071859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.072216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.100127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:51.114560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:51.115631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:51.116676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:51.119239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 212/416] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:51.123891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:51.208550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b58a0a-5d9a-4fa0-8136-63f9ea6f5d34 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:51.273753 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.274664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.274955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.275234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] PATH_INFO: `/v3/projects/0064d8382e454dc1bf5476c00fe5a47a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.276956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] RBAC: Authorizing `identity:get_project(project_id=0064d8382e454dc1bf5476c00fe5a47a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:51.279174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:51.281367 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 205/417] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:51 2022] GET /identity/v3/projects/0064d8382e454dc1bf5476c00fe5a47a => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:51.288465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:51.342410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc93b3-435c-47a8-98c4-ca4c183751ee None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:51.383680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:51.390026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:51.407061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.408140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.408685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.409265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.410356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:51.410904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:51.412578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18654f89-d34a-40e6-a623-52974e6bb107 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:51.424173 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 213/418] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:51.426181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fe4b-96f6-4b77-b279-b5f2263ec5dd None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.426751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fe4b-96f6-4b77-b279-b5f2263ec5dd None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.427374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fe4b-96f6-4b77-b279-b5f2263ec5dd None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.429660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 214/419] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:51.430737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1fe4b-96f6-4b77-b279-b5f2263ec5dd None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:51.449644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9abd2f-0673-4417-8b30-aef7723714b5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:51.490272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.491174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.491463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.491745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] PATH_INFO: `/v3/projects/0064d8382e454dc1bf5476c00fe5a47a/groups/1dd049a94e1a4bb1b0b11a7153b9397b/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.493933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] RBAC: Authorizing `identity:create_grant(project_id=0064d8382e454dc1bf5476c00fe5a47a, group_id=1dd049a94e1a4bb1b0b11a7153b9397b, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:51.495873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051243c-21be-4f58-8d3e-8e52f9e6826c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:51.520509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 206/420] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:10:51 2022] PUT /identity/v3/projects/0064d8382e454dc1bf5476c00fe5a47a/groups/1dd049a94e1a4bb1b0b11a7153b9397b/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:10:51.520509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.520509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.520509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.546950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:51.548214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:51.633031 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 207/421] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:51.661835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1fe4b-96f6-4b77-b279-b5f2263ec5dd None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:51.675894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:51.742146 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.743233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.743767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.744263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.750826 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:51.818891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:51.830557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff010c-7595-4ae5-bc71-0c1f9bbac529 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:51.846318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:51.848073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:51.868269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:51.869574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:51.870822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:51.871529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:51.883963 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:51.892801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:51.893870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:51.894786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:51.898135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 208/422] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:51.906412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.052343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 215/423] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:52.057956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.082452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3b025-8cc1-4bd3-b3ba-bc2df85c321b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.112779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.122392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.123502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.124176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.124715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.125701 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:52.126330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.128040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.138202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 209/424] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:52.143601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.221115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.253256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf904cb-acdd-4f70-a6dd-7217a0eb25e7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.316230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17148321-f648-4dbe-b1d3-b55be848882f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.355480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.356741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.357265 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.357779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.381731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.382629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.382917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.383197 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.384499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.386207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.388231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 216/425] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:52.393262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.395452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.440231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.442304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.565755 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 210/426] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:52.572777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.648874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.691429 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378064-6353-4ac2-81d3-e7018e666fb4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.722993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04645490-4073-47b9-80d5-d6bf289f4105 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.734455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.735591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.735995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.736512 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] PATH_INFO: `/v3/projects/18c9cf67e140413ea746a42df18a6f98` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.737867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] RBAC: Authorizing `identity:get_project(project_id=18c9cf67e140413ea746a42df18a6f98)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.739551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3662f-c1c2-4bdb-94ea-946d2bfa812d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.741831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 211/427] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/projects/18c9cf67e140413ea746a42df18a6f98 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:52.748498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f0d3-2720-4e98-a1ba-936044cfb70b None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.748899 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f0d3-2720-4e98-a1ba-936044cfb70b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.749285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63f0d3-2720-4e98-a1ba-936044cfb70b None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.751190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 212/428] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:52.752001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63f0d3-2720-4e98-a1ba-936044cfb70b None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.762419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.763287 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.763570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.763854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.765106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.766680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed478e-bc14-48b8-8416-fe49b5cad9b3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.768608 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 217/429] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:52.773894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.774191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.774474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.794514 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:52.795724 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:52.808789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.872817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63f0d3-2720-4e98-a1ba-936044cfb70b None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:52.873137 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 218/430] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:52.874077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.922663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:52.930150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:52.932350 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:52.933920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:52.934489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:52.936497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:52.938442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:52.940813 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 213/431] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:52.941996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:52.994657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1223ead-7818-4c02-95f6-ff83ade8ae15 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.037122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.038210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.038210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.038397 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.039211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=18c9cf67e140413ea746a42df18a6f98) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:53.039529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.041084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.047858 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 219/432] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=519d740b9c3d46f3902513905864fa5d => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:53.052714 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.061626 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d00fb-9db0-4205-88ba-fcbce1a95c10 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.077895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.107486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.109135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.109674 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.110188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.124066 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d7c1adbe-19bd-4c34-80a5-8db406d7aac6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:53.132823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:53.133897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:10:53.134793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:53.138422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 214/433] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:53.145062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233f85-2d4e-47c6-aa0f-053192d30d8d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.146049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233f85-2d4e-47c6-aa0f-053192d30d8d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.146555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233f85-2d4e-47c6-aa0f-053192d30d8d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.149041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 215/434] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:53.150130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5233f85-2d4e-47c6-aa0f-053192d30d8d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.202919 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbede3b8-4f27-41f8-bca8-6d89723f0c30 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.240886 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.241147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.241666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.241954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.242235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.243123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=18c9cf67e140413ea746a42df18a6f98) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:53.244092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.245830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1607c-f88a-423e-afdb-ce426a4cb595 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.253879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 220/435] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=18c9cf67e140413ea746a42df18a6f98&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:53.255212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.255504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.255787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.276630 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:53.278010 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:53.290577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5233f85-2d4e-47c6-aa0f-053192d30d8d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.333039 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.334177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.334695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.335208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.336212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] RBAC: Adding query filter params (name=member) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:53.336892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.339122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.350956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 216/436] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:53.358162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.379426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 221/437] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:53.404956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.408335 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.484590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f358e-70ae-4b94-9d11-10423e6754bb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.513937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.530814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.531162 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6c1fe-1558-49e5-bb62-4acd5cdc7a97 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.531484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.531776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.532056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.584678 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.585520 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.585810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.586094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.601327 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:53.612364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:53.613443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:53.614318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 222/438] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:53.617391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.633477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.636532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.707504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.764866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54bacb-8607-4da9-a444-1ad88ffdeef4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.770846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 217/439] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:53.778378 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.813606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.813996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:53.814656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.814965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.815256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.816228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:53.816484 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.817875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.828090 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 223/440] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:53.833836 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:53.934444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247af2f3-e75a-4aeb-90d0-fdd0891a9479 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.983402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae55550-34be-438c-972f-3ce7154a73f6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:53.983743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:53.984738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:53.985031 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:53.985316 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:53.987196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:53.988745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:53.990584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 218/441] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:53.998396 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.024983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.039307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.040170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.040923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.041130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.123572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.125458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b191-c96d-4f8b-b2b2-7fff9fe276c7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.131963 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.151802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba11d-2add-4151-8eb0-3595bede3730 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.209600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.210465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.210751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.211028 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.212295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.214406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc50ae0-2e56-42db-bb18-62b25e4f21b5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.216266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 219/442] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:54.217487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b612c-a847-4e32-9c8a-adb6e01f0072 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.217778 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b612c-a847-4e32-9c8a-adb6e01f0072 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.218057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b612c-a847-4e32-9c8a-adb6e01f0072 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.219435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 220/443] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:54.224804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b612c-a847-4e32-9c8a-adb6e01f0072 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.258237 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 224/444] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:10:54.259627 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.259931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.260215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.279551 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:54.280809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:54.304517 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.340782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b612c-a847-4e32-9c8a-adb6e01f0072 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.362865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 225/445] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:54.363729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.388350 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.389077 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.389353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.389631 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.390481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=dc20953902b3457186b2eb9d4ec05d6b, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:54.390796 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.392315 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.398898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 221/446] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=dc20953902b3457186b2eb9d4ec05d6b => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:54.399742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.437090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.487465 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768dfb3-34b9-44d1-9c0c-e8dd2777596d None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.532172 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.532432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba206b-12c1-4f9e-87df-a50bcee38992 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.533099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.533391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.533666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] PATH_INFO: `/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.536029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] RBAC: Authorizing `identity:get_project(project_id=f707fa233ec64e4a8da5e1ce11fd2786)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.537579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.545117 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 226/447] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/projects/f707fa233ec64e4a8da5e1ce11fd2786 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:54.546081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.575221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.576342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.576679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.576959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.578552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.593717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:10:54.603784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:54.606630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:54.607536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 222/448] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:54.611468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.674236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f7bf3-7fb9-49f5-a984-26a7b75738ed None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.712192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.719142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.719930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.720210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.720528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] PATH_INFO: `/v3/groups/b6d02a35ddb042ff89015301bbf3541a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.722523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] RBAC: Authorizing `identity:get_group(group_id=b6d02a35ddb042ff89015301bbf3541a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.723971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.725692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 227/449] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/groups/b6d02a35ddb042ff89015301bbf3541a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:54.732199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.743560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec509df6-a38e-4424-9cda-0f318ab7b1e0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.789434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.791421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.791421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.791421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.791421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:54.791779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.793208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.812574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 223/450] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:10:54.818027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:54.840629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:54.849634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11afb82a-7103-45b1-a958-36ac0163dbe9 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.886845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.887568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.887858 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.888142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.889051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] RBAC: Adding query filter params (group.id=b6d02a35ddb042ff89015301bbf3541a, role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:54.889376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:54.890843 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:54.897072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 228/451] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/role_assignments?group.id=b6d02a35ddb042ff89015301bbf3541a&scope.project.id=f707fa233ec64e4a8da5e1ce11fd2786&role.id=519d740b9c3d46f3902513905864fa5d => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:54.927605 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081051cd-acef-4f32-b835-d2aed3fb60b1 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:54.966262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:54.966947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:54.967342 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:54.967653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:54.984991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:55.044756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:55.046153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:55.154674 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 224/452] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:10:55.160554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:55.275373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0a528-99fe-44e0-a00a-638778905558 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:55.318616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:55.319406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:55.319692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:55.319966 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] PATH_INFO: `/v3/projects/0064d8382e454dc1bf5476c00fe5a47a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:55.321181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] RBAC: Authorizing `identity:get_project(project_id=0064d8382e454dc1bf5476c00fe5a47a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:55.322632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:55.324432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 229/453] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:10:55 2022] GET /identity/v3/projects/0064d8382e454dc1bf5476c00fe5a47a => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:55.330343 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:55.376495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:55.433024 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0dbb5-e054-4f05-b5d0-fdf4aa1a7bf4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:55.469136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:55.469994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:55.470276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:55.470551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] PATH_INFO: `/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:55.471712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] RBAC: Authorizing `identity:get_group(group_id=1dd049a94e1a4bb1b0b11a7153b9397b)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:55.473185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e3eaf-b27e-47c4-a007-f847b504b8c1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:55.474783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 225/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:10:55 2022] GET /identity/v3/groups/1dd049a94e1a4bb1b0b11a7153b9397b => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:55.481456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:55.506885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:55.582319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee1a4-2980-4759-8ceb-608f9015a263 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:55.617762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:55.618417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:55.618703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:55.618981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:55.619961 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] RBAC: Adding query filter params (group.id=1dd049a94e1a4bb1b0b11a7153b9397b, role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=0064d8382e454dc1bf5476c00fe5a47a) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:55.620285 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:55.621762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbf898-b499-409e-91b0-415168b24c98 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:55.627469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 230/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:10:55 2022] GET /identity/v3/role_assignments?group.id=1dd049a94e1a4bb1b0b11a7153b9397b&scope.project.id=0064d8382e454dc1bf5476c00fe5a47a&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:58.392639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c421c-9dc8-4966-b25a-408a80273298 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:58.393197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c421c-9dc8-4966-b25a-408a80273298 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:58.393197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c421c-9dc8-4966-b25a-408a80273298 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:58.394892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 226/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:58.400702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85d12-c441-428a-aec4-7cc313e20af6 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:58.401750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85d12-c441-428a-aec4-7cc313e20af6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:58.402078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85d12-c441-428a-aec4-7cc313e20af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:58.433975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-79e85d12-c441-428a-aec4-7cc313e20af6 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:58.435219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-79e85d12-c441-428a-aec4-7cc313e20af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:58.547157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 231/457] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:10:58.595838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c421c-9dc8-4966-b25a-408a80273298 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:58.771134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c421c-9dc8-4966-b25a-408a80273298 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:58.786587 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:58.822134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:58.823451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:58.823731 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:58.824013 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:58.838221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5ff2-fc7a-4001-835e-392b20cec102 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:58.839272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5ff2-fc7a-4001-835e-392b20cec102 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:58.839631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf5ff2-fc7a-4001-835e-392b20cec102 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:58.841586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 232/458] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:58.843236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:58.844837 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:58.848859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:58.849491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:58.850043 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:10:58.864691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:58.865819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:58.866957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:10:58.866957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:10:58.866957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:58.866957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:58.866957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 227/459] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:58.876958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:58.887695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:58.889338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:58.906068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:58.969734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 233/460] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:59.005191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baa64a-454c-4884-bdd0-7e7af432ca85 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.017211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.045869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.046625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.046921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.047210 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.048015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:59.048334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.050034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.065923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 228/461] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:59.075304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.106952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:59.197275 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e523e-0c95-4d4c-8918-db94b16643b2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.223002 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e892a-cede-4214-853e-69222cc989d8 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.238962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.239660 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.239948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.240228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.241234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.243440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0bf8c-3b61-4417-962d-f944cb71374e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.270581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.271624 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 229/462] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/users => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:10:59.272471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.273391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44925-cbb4-4f24-b66f-22454d3b9c54 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.274150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44925-cbb4-4f24-b66f-22454d3b9c54 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.274680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde44925-cbb4-4f24-b66f-22454d3b9c54 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.276940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.281480 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.281746 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 230/463] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:59.283642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c74d-f912-43d1-b37c-38d16945f167 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.284090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c74d-f912-43d1-b37c-38d16945f167 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.284549 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82c74d-f912-43d1-b37c-38d16945f167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.299785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.301348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.305792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-fb82c74d-f912-43d1-b37c-38d16945f167 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:59.307862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-fb82c74d-f912-43d1-b37c-38d16945f167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:59.320652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:10:59.342413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:59.343567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:59.344706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:10:59.344706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:10:59.344706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:59.344706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:10:59.344706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 234/464] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:10:59.351140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.398713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 231/465] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:59.482874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e22d-41c5-4c73-bf47-9a7ede2745c9 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.483206 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e22d-41c5-4c73-bf47-9a7ede2745c9 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.483440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7e22d-41c5-4c73-bf47-9a7ede2745c9 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.485442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 232/466] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:59.486860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e22d-41c5-4c73-bf47-9a7ede2745c9 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.503159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:59.535442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd24f89-d3ef-4e88-ac93-7f6ec4b51eb5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.584221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.584221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.584221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.584623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.585173 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:59.585485 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.587132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb925884-6da5-4e8b-a95f-dc10b66301aa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.601552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 235/467] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:59.603369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.603779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.604195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.624875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:59.626190 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:59.634000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7e22d-41c5-4c73-bf47-9a7ede2745c9 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.680285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.681150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.681688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.681965 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.693404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.694950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.696157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:59.711294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 236/468] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:10:59.712167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:10:59.719959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:59.721029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:59.721970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:10:59.721970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:10:59.721970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:59.721970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:10:59.721970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 233/469] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:59.733706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.816503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:59.835188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d4d1-6e01-4fd2-9a46-7239f2885ffb None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.866255 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849a0a4-7282-4301-a6f1-1e3e2c2a4007 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:59.877611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.878449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.878746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.879034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.879871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.881388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.907236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 237/470] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:10:59.908200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.908818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:59.909603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.909901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.910182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.911207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:59.911532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:59.913191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:59.935269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 234/471] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:59.936830 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:59.978591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.066392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dceca67-5026-41ed-be18-540c44fb73e4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.068584 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964ce74-e70e-45ac-a88d-2d1af90634d5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.094199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.108371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.109238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.109353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.109644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.110379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.111859 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666e139-8571-492e-8e38-e0567d82ab1d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.127962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.127962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.127962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.128307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.146057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 235/472] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:11:00.152227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f647-637d-4e7b-b3a7-e8df52dd7429 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.152227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f647-637d-4e7b-b3a7-e8df52dd7429 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.152227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3f647-637d-4e7b-b3a7-e8df52dd7429 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.152640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 236/473] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:00.158388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.160020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.162095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.162389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.162669 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.196215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:00.197541 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:00.203337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:00.204389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:00.205365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:00.205365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:00.205365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:00.205365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:00.206088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 238/474] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.214393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.317734 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 237/475] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:00.349672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.353550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.355484 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a823d7-c0ad-4bd4-b1bb-d5eb7d3f55f7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.409166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.409896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.410233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.410490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.411317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:00.411574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.413190 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.427733 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 239/476] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:00.436271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.503978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75188d41-3f23-4b4a-83a4-94801e42bda9 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.518174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.555219 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.555899 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.556183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.557499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.558443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.560428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.574247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.sql.core [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:11:00.574247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:11:00.574247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [parameters: {'id': 'bf229961cb5f4c63890b4f4935eea245', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:11:00.574247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214148) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:11:00.575856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854092c-b612-4fbb-b5e0-1730a365c845 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application conn.query(q) May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result.read() May 19 20:11:00.594846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:11:00.595868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application next(self.gen) May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application yield resource May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application next(self.gen) May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise self.value May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application session.commit() May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:11:00.596946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.session.flush() May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application compat.raise_( May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:11:00.597952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:11:00.598935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application conn.query(q) May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result.read() May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:11:00.599847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [parameters: {'id': 'bf229961cb5f4c63890b4f4935eea245', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.602016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:11:00.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.614119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 238/477] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:11:00 2022] POST /identity/v3/roles => generated 144 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.614119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.634448 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.636097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.636384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.636687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.643547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.643547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.674997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 240/478] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:11:00 2022] POST /identity/v3/users => generated 249 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:00.758301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5905c74-3606-4353-9c2d-8900036c190b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.799912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.801211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.801817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.802098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.816099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fe29b445-3a09-4ab8-9905-7594d33b1271 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:00.825337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:00.827787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:00.829135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 239/479] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.834320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.864848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.965194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27907a9f-5d3a-4d88-8a28-aa65fc9def7e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.002915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.003652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.003941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.004222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.005039 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:01.005351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.006807 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82937d0-2de0-4d14-a34b-7d5fae5458b3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.017129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 241/480] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:02.223009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad055-bd71-4397-8640-2c06f172eaae None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.223611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad055-bd71-4397-8640-2c06f172eaae None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.223756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad055-bd71-4397-8640-2c06f172eaae None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.225513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 240/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:02.232215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.232854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.233001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.257799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:02.259025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:02.370940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 242/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:02.418414 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ad055-bd71-4397-8640-2c06f172eaae None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:02.462769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:02.577234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ad055-bd71-4397-8640-2c06f172eaae None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:02.624891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:02.626105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.626617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.627131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.645511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-c53e8281-4fc0-4370-b7c1-64985d83188c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:02.654199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:02.655266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:02.656216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:02.659267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 241/483] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:02.664965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:02.679781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:02.861645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397e170-394d-4a0a-8479-a499ad5b822a None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:02.912978 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:02.913689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.913982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.914268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.915138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:02.915455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:02.917493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:02.933318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 243/484] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:02.933512 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7816b9c-f424-4c3d-9feb-15867fa9f98d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.933723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7816b9c-f424-4c3d-9feb-15867fa9f98d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.933926 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7816b9c-f424-4c3d-9feb-15867fa9f98d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.935663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 242/485] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:02.940730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:02.941742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.942273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.942764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.962276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:02.964673 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:03.057703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 243/486] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:03.076613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.124576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.148437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26740926-d69b-4a80-a7db-995faa8d1cb5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.173175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.220794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.221639 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.221936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.222227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.244037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.246194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:03.278640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:03.279659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:03.280642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:03.280642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:03.280642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:03.280642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:03.282203 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 244/487] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:03.285940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.327792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311a896-fa0a-4ae1-982a-4756ee15481e None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.363947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.364777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.365066 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.365344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.378029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-1bc4d9aa-f3a1-4ec1-8e55-cdf6729699ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:03.385997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:03.387086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:03.388011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 244/488] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:03.389781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad886f-4779-4bc3-92fa-6d23b3118c32 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.389781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad886f-4779-4bc3-92fa-6d23b3118c32 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.389781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad886f-4779-4bc3-92fa-6d23b3118c32 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.392193 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 245/489] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:03.392972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad886f-4779-4bc3-92fa-6d23b3118c32 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.429277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a0e69-03a6-45be-ae4f-a819863bd385 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.461646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.472975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.473664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.473951 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.475034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:03.475400 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.477861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69254d8c-7f00-4463-9362-e2dd29389215 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:03.500076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 245/490] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:03.501548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.501850 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.502575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.520086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:03.521017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.521390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:03.549574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad886f-4779-4bc3-92fa-6d23b3118c32 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.592714 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.593779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.594358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.594799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.595878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:03.596469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.598063 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:03.601346 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 246/491] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:03.601809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.609363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 246/492] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:03.616585 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.745363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f63e3-1762-47ea-894c-ee30ffd75146 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.749075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262fb675-a427-43fa-bbca-38ba2f9c8af5 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.760952 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.784298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.785131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.785424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.785705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.787570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.788435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.788720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.789014 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.797622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.799113 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:03.799232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.800633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:03.803638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:03.816999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:03.818215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:03.819325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 247/493] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:03.819995 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-81ccc9c5-4182-4a05-8df8-f6dd8106cc96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:03.825788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:03.826819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 247/494] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8210fe-7e49-4816-b0a6-7c1479ee6b5c None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8210fe-7e49-4816-b0a6-7c1479ee6b5c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8210fe-7e49-4816-b0a6-7c1479ee6b5c None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.834129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 248/495] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:03.835552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8210fe-7e49-4816-b0a6-7c1479ee6b5c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:03.943889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8098b-60d3-431b-8e75-3e01830b4eba None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.945358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8210fe-7e49-4816-b0a6-7c1479ee6b5c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:03.986439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.986981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:03.987281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.987574 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.987862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.988355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:03.988828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:03.989256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:03.990287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:03.990781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:03.992479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62976843-b4f5-42da-83c5-a6cf1b19b4d0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.001913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:04.009897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.011300 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 248/496] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.017254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.043283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 249/497] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.043283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.043283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.043283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.059238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:04.060494 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:04.064204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.117407 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.137802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 250/498] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:04.138645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.162205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2b675-03ee-438f-b0d3-688f61d9e92f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.200578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.201256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.201542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.201819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.202563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:04.202873 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.204375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.218015 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 249/499] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.218883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.248034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bef78-e884-42e8-b644-4ad46d715307 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.292751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.293340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.293639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.293916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.294677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:04.294997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.296429 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.304412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 251/500] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.305243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.322078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.333266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01477083-9c0d-4baa-a3d4-6e04c028fb56 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.355829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.369213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.370626 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.371281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.413803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76671-9421-4f2f-a617-bb27b93a1e23 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.442909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.444389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.451335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.452133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.452441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.452838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.463879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d776c15d-206d-42aa-b954-46855c66cd81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:04.472004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.473388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:04.478215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 252/501] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.479565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149442f-8b4a-46e2-a2d9-3c4b85119c15 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.479861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149442f-8b4a-46e2-a2d9-3c4b85119c15 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.480147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149442f-8b4a-46e2-a2d9-3c4b85119c15 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.481501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 253/502] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:04.482205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149442f-8b4a-46e2-a2d9-3c4b85119c15 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.544912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.555428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 250/503] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:04.560179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.589818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149442f-8b4a-46e2-a2d9-3c4b85119c15 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.596857 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.627860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.627860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.627860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.628290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.636755 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.638223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:04.655050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.656194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 254/504] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.662311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.667775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a05df2-596a-42ef-9db2-e4b71ca1d2c4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.709067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.709872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.710239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.710445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.722639 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.724288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d82d88c6-9e36-40c0-8c67-0e75575b089a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:04.744708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.745934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.747241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:04.747241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:04.747241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:04.747241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:04.747241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 251/505] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.747701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.748000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.748289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.765272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:04.766497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:04.769564 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275708b-9dca-4690-ac5a-48429350ddf4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.806036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.806878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.807282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.807742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.808533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:04.808861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.810255 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.811875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.819231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 255/506] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.820386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.846545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 252/507] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:04.847417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.854645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.930164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6388b-4146-4987-b0d3-828536fb24b7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.958093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1548c-aa29-439e-a652-fb8f20f810e2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.974211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.975151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.975324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.975607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.988078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.989716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.993325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.994027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.994317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.994690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.995559 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:04.995871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.997370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:05.007685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.008819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.009887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:05.009887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:05.009887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.009887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:05.009887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 256/508] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.010713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.037057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 253/509] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.037969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.071136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.101037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.121069 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a363b-1247-4fef-9c4b-5c7a3b1a96e5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.145633 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf0af3-53f6-4597-b95b-347c67c7b390 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.156747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.157368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.157651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.157938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.158677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.158991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.160420 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.172422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 257/510] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.177323 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.181902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.183242 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.183524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.183802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.197730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.199722 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:05.218845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.220489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 254/511] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.222365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.307463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.310387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efebe5-042b-416d-ad27-3cc9d2eabb8d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.327983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81d45a-9f24-438d-b733-fbe979e62e22 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.345497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.346290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.346574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.346853 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.356263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.359900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.361569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.369954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.370802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.371094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.371384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.372130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.372482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.373883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:05.383249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.384188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 258/512] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.385844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.390812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 255/513] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.391701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.493170 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc814a5-973f-4ddf-93a9-0d0e3654c4de None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.511155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41600b10-39c3-47ec-96dd-618d52e453a8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.529958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.530627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.530915 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.531198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.553959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.556132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.556686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.556970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.569202 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.570696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:05.586875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.588142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 256/514] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.590020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.593471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.608091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.618713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.620882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.700045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43566790-125c-40c8-9eb0-17e47152d0c6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.731102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 259/515] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:05.731873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.742740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.743419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.743712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.743997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.744811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.745130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.746540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.758567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 257/516] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.759594 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.825406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.835903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.846628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5969-e3de-4687-ba86-cf533ff14024 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.876640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de91d-26cf-4a01-935e-f23bd503cba0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.888695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.889360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.889644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.889918 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.890648 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.890952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.892319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.904920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 260/517] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.905739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.914066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.914877 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.915165 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.915447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.923632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.925175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:05.941499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.942716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 258/518] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.943706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.031603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476d42a-4ef3-437d-8afe-b681e6928bd2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.049380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd5d4b-98d4-4f21-ac12-96a3475806c4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.066238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.066916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.067202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.067482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.068331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.068680 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.070128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.071896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.085105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.085912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.086198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.086484 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.091043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 261/519] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.091814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.097924 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.099352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.104818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:06.117800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:06.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:06.119738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:06.119738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:06.119738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:06.119738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:06.119738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 259/520] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:06.120048 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.230067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee2e2-8525-4762-b2a5-2eaafc8289c9 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.231118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc98e-cd74-4f6a-ab1e-cb964c27edf0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.266295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.267091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.267382 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.267672 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.272378 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.273086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.273374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.273657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.274380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.274909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.276597 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.276991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:06.292516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:06.293798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 262/521] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:06.294817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.309301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 260/522] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:11:06 2022] POST /identity/v3/users => generated 247 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:06.310203 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.364339 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.389414 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.432991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd91e5-d5b6-4382-ac04-02c6ecbe9969 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.455372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0abdc1-589b-4934-9df7-9de96ba1ea9b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.477313 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.478004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.478302 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.478579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.479659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.480160 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.481705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.497527 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.498216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.498510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.498798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.510159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 263/523] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.511574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.588012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.589768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.627089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.642706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd9cf7-52d4-4b4f-81f3-38db60c7962e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.679004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.687095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.687780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.688095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.688352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.689125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.689444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.690842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.707361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 264/524] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.708791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.720188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 261/525] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:06.722271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.855743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dd489-191e-4ce9-bf4a-5faf9199bca1 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.857215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435c0a6-e8b9-4610-a914-36e797315180 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.892600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.895789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.896722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.897192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.897477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.898333 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.898655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.899773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.900126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.900467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.900783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.901200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.902057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.902382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.903837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.912557 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 262/526] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.913438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.931243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 265/527] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.932734 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.996094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.049909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59465438-7f6e-40e6-a82b-5bdc84178093 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.075556 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e8a31-348d-4ee7-a634-f8175e983185 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.092631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.093845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.094407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.094910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.103658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.105734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:07.122854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:07.123892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:07.124894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:07.130988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.130988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.130988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.131426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 263/528] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:07.131426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.131579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.147069 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.149030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:07.169499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:07.170529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:07.171477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:07.171477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:07.171477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:07.171477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:07.171477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 266/529] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:07.171786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.214230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.281526 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4cc98b-c97b-42b8-b467-2c0791a5eaf8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.294051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.297277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb668e28-2ed7-484e-9cc4-dbd4b4c0ce51 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.329447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.330464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.330984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.331563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.336931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.337410 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.337780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.338133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.345154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:07.345154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.345154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.355817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 267/530] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:07.356903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.420545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.422782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.466354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b77cf-be81-44a2-8f6f-153cd79f09d2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.489831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.503653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.504485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.504954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.505146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.506257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:07.507110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.508149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.522774 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 268/531] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:07.523921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.546892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.556040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 264/532] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:07.558308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.635571 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5ff1b-99a4-44c0-8fc6-3b407c38f487 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.669646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51478b9d-bbc7-4cc3-a0da-6bcf1d8e4c9f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.681301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.682315 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.682682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.683214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.684629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=65534c37a2b8445c8151d802ad980e13) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:07.685026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.686910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.694868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 269/533] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/role_assignments?user.id=65534c37a2b8445c8151d802ad980e13&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:07.696590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.717610 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.718887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.720148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.720148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.730011 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.731956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:07.748290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:07.749331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:07.750295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:07.753092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 265/534] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:07.758991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:07.787256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.876800 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:07.895796 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97afafe-8d4a-464d-ac81-de1682942597 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.945840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:07.946570 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:07.946862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:07.947172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:07.957911 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63056d7a-32c5-4521-9939-6726385d69df None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:07.959816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:07.961453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:07.988426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:07.989457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:07.990402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:07.990402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:07.990402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:07.990402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:07.990402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 270/535] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:07.991291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.015746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.016909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.017427 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.017941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.018981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:08.019533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:08.021512 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:08.043202 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 266/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:08.049697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.118181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0b88b-b66f-4466-9708-d66f8ff11728 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.163047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.163735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.164027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.164314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.165594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:08.165919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:08.168301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:08.171005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:08.184478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 271/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:08.191545 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.208434 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:08.260613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698bbc7-aeab-47cb-8daf-ba6872d10f3e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.315394 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6114b6-5f19-4a73-93aa-fb3d4402e915 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.319412 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.320093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.320382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.320699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.321560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=9f436f3923374b2b97fd95d05bd884f4) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:08.321882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:08.324102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:08.330957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 267/538] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/role_assignments?user.id=9f436f3923374b2b97fd95d05bd884f4&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:08.403552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.404386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.404696 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.404986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.421993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:08.423581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:08.446840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:08.448469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:08.449531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:08.450958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 272/539] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:08.456866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.686242 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7354e-0c28-4e85-b6b8-c96ef0844d4c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.753760 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.754820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.755346 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.755846 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.756904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:08.757458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:08.759174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e629d5d-4069-4896-8fef-08f4d3573b82 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:08.776464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:08.788072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 268/540] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:08.801853 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.897497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f44b-755b-40c3-aa0c-1ce5e5a9443f None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.897849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f44b-755b-40c3-aa0c-1ce5e5a9443f None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.898065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90f44b-755b-40c3-aa0c-1ce5e5a9443f None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.899998 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:08.900182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 269/541] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:08.906271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.907823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.907823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.958304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d03876-86e4-4082-af4d-342670f46f00 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.966628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:08.967880 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:09.038409 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.039041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.039337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.039622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.040501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=2aa1864ac1e1481a9c09f418e9978944) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:09.040824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.042860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.064873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 273/542] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/role_assignments?user.id=2aa1864ac1e1481a9c09f418e9978944&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:09.121110 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 270/543] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:09.167678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.385622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5083-ee13-413a-b94d-d8849ca61ca5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.415084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.440050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.440900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.441192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.441478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.458170 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:09.466767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:09.467807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:09.468769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 274/544] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:09.474076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.627029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf7cbf-ad31-4a79-b4dc-5505691964e5 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.675559 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.689273 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.689987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.690275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.690556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.691348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:09.691657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.693302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.703110 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 271/545] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:09.713597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.862544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cbcfc-7fbb-48d9-a900-496e51983c6b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.911985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.912818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.913393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.925872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.927263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0497fd2f-8e1d-4862-9a9f-068641d0101e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:09.955603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:09.956678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:09.957629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:09.957629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:09.957629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:09.957629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:09.957629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 275/546] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:09.962498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.146091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d0d6-7f78-4475-b6e0-3f1b95ec7008 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.197331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b385-0bf3-4ceb-bcd4-a7766e83b520 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.197595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.198110 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.198399 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.198708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.198987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b385-0bf3-4ceb-bcd4-a7766e83b520 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.199739 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.199833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.200446 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378b385-0bf3-4ceb-bcd4-a7766e83b520 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.202599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 276/547] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:10.206895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.208747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.209283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.209768 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.222545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 272/548] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:10.232453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.247043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:10.248528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:10.299846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.320541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.383049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 277/549] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:10.406153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.422988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006ce2-71e2-414a-a0f0-1dc916e7b52a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.500177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.500977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.501245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.501525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.513016 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.514785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:10.544257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:10.545488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:10.546497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:10.560598 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 273/550] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:10.568644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.597911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd526c8-0827-41fb-af6b-c3a82367f959 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.678903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.679726 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.680015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.680298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.695036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-dfdc7c5d-6022-4a96-be87-fe6a6a9f7df1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:10.705450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:10.706492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:10.707388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 278/551] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:10.712931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30137-53fe-4af3-a12b-ac5cd9c94716 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.712931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30137-53fe-4af3-a12b-ac5cd9c94716 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.712931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc30137-53fe-4af3-a12b-ac5cd9c94716 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.712931 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 279/552] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:10.718499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc30137-53fe-4af3-a12b-ac5cd9c94716 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.724227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48986d4-208c-4a82-a7de-8b040c1b89a0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.745007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.749805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.771868 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.773019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.773529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.774070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.775228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.776846 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.778696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40217748-da75-48df-b7a6-6ae2b920a31f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.801325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 274/553] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:10.803178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.803706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.804203 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.825068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:10.827331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:10.853210 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc30137-53fe-4af3-a12b-ac5cd9c94716 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.898167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.899675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.900289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.900866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.902017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.902718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.904499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.920595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 275/555] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:10.920595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.921123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 280/555] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:10.922325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.050750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.055327 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728874df-e458-45dd-a59b-c4cc981404c2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.065355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.087422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856e436-895d-40d5-8188-6ba784a92599 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.098899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.099775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.100142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.100546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.126334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.127903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.128684 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.129161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.144097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.145974 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:11.164865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:11.165907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:11.167460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:11.167460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:11.167460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:11.167460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:11.167460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 281/556] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:11.168238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.190342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.192239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e178ff-21b4-4884-86c3-d789c04d861f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.289460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3836f-f3b8-4d6f-be21-e9a345f62375 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.306098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.316941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 276/557] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:11.317970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cafd1f-592c-4010-bc2a-e6f4e714b34c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.318266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cafd1f-592c-4010-bc2a-e6f4e714b34c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.318558 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cafd1f-592c-4010-bc2a-e6f4e714b34c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.320020 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 277/558] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:11.325267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cafd1f-592c-4010-bc2a-e6f4e714b34c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.333692 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.334483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.334770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.335046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.339453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.350942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:11.359713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:11.360887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 282/559] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:11.361812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.446786 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cafd1f-592c-4010-bc2a-e6f4e714b34c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.480293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7472d-729a-48b1-b120-e51a1f175a9c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.486853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.487549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.487838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.488129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.488932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:11.489250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.490702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1527fe7-1e6f-45da-8e26-b7bafd1a3f23 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.504556 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 278/560] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:11.505987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.506352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.506859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.524761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:11.526662 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:11.528331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.529496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.530062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.530346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.542802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.544311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:11.564852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:11.565855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:11.566822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:11.566822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:11.566822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:11.566822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:11.566822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 283/561] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:11.567137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.591523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.612450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 279/562] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:11.612450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.632847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.699742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ecf0d-1620-4be1-af16-6caa9354270b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.735563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.736243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.736562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.736845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.737607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:11.737917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.739313 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.740885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f051a-222b-4e4e-9600-f12888532a42 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.747499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 284/563] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:11.748363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.782542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.783533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.783925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.784197 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.791547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.793062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:11.808838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:11.810091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 280/564] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:11.813998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.859228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab05df-764c-49b3-821d-7782f8021b87 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.870341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.874546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:11.899537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.900196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.900516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.901452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.902187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:11.902503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:11.903877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a8259-dfd6-4fd9-888c-3996e8f81684 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:11.916145 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 285/565] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:11.918226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f0a2-bce1-42d5-85aa-190251c8e41a None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.918523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f0a2-bce1-42d5-85aa-190251c8e41a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.918802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f0a2-bce1-42d5-85aa-190251c8e41a None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.920472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 286/566] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:11.921214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4f0a2-bce1-42d5-85aa-190251c8e41a None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:11.929650 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8d0a0-4ff7-4e8a-af47-57c450243097 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:11.973303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:11.974246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:11.974531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:11.974811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:11.987957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:11.996667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:11.997856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 281/567] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:11.998848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.030492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4f0a2-bce1-42d5-85aa-190251c8e41a None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.067290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.068176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.068495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.068779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.081007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.082508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-736def5e-871a-41e0-8684-29b10fb71d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:12.100789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.101945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.102901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:12.102901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:12.102901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.102901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:12.102901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 287/568] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.103565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852f2c4-41c3-48eb-94c5-8988a3566929 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.103696 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.103994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.104512 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.115155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.121964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:12.123293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:12.141945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.144573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.145260 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.145547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.145827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.146692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:12.147002 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.148474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.166904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 282/569] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:12.167756 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.194688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 288/570] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:12.195588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.294529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a02293-60da-4c19-8b4e-22f2f9f7ba35 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.307519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8dfe9-0596-4d0a-b0fe-d2acbacdd6e1 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.329694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.330498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.330789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.331076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.339415 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.340962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.343180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.343865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.344153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.344509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.345245 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:12.345561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.346986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:12.355980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.357047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:12.357991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:12.358455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 289/571] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:12.358455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.358598 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 283/572] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.358598 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.375354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.390637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.467999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33a76d-e8ef-476f-87f9-9e0431e1a33d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.470644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fe6fd-bacc-4a3d-86b3-a19436baa633 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.502641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.503445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.503728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.504006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.507161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.507795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.508086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.508363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.509137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:12.509449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.511364 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.515938 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.517476 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.523481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 290/573] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:12.524305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:12.540388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.541530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 284/574] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.552483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.610853 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.635954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.640200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bbac7-78d4-4de4-87ac-7db1454e0574 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.661162 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53b867-b4fd-4263-8698-75e9bf8ee182 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.677508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.678317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.678599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.678875 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.690088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:12.698537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.700487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:12.702792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:12.704308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.704532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 291/575] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.704532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.704831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.704831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.704831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.720304 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.722723 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:12.747971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.749207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.750369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:12.750369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:12.750369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:12.750369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:12.752588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 285/576] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.753741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.824589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2d70-7b08-4eed-b7eb-8b7a239d0fb8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.866230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac6bd4-ee60-48c5-8040-f57d0113b42c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:12.868948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.870166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.870540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.870822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.872291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:12.872874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.874705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.877209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.892164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 292/577] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:12.893132 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:12.898575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:12.905850 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:12.913237 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:12.913591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:12.913807 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] PATH_INFO: `/v3/users/glance` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:12.923715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:12.925174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:12.947056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:12.948483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:12.949767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 286/578] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:12.950568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.024270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d6f5-c787-4119-9e93-e6e318da80fa None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.062079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.062765 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.063068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.063359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.064152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.064507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.065947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.085247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 293/579] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.085499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611d928-b11d-463e-bb47-321028c9cd8a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.086548 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.126898 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.127595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.127885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.128166 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.128981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.129296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.130698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.142323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 287/580] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.143786 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.164548 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.212261 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.218876 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25f70e-f5ee-4574-85b2-b9203bee2622 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.261761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.262446 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.262757 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.263050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.263780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.264091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.265537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.281546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0699169-d4ea-4d9c-af3f-1b8c5f4bccc8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.285565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 294/581] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.286505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.324139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.324964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.325324 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.325693 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.326708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=1037a6248c8241828857d69fba1dce7d) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.327044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.328795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.339457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 288/582] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/role_assignments?user.id=1037a6248c8241828857d69fba1dce7d&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.340727 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.422537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d6f28-5ac2-49f7-8746-0bbf4d31c260 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.442735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.471675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.472370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.472686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.472966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.473825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.474141 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.475659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.488673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a89c9-c48c-4fbf-b2e5-d8844f5f39cb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.501240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 295/583] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.502153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.515617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.556256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.556256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.556256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.556256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.571730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.575748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:13.598407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.600358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:13.602135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 289/584] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.606277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.647248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b27b0-bee9-425d-b9d6-c17d16fb17ec None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.687274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.687997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.688270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.688577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.703668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.705244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:13.724155 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.725381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 296/585] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.726618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.733725 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336840-f9c9-4a2d-814b-4aa14a9b3046 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.759217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.774350 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.775289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.776297 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.776728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.784858 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.786475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:13.802532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.803991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:13.805094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 290/586] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.805729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.849740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.857978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0a456-fc40-436a-89c0-222082af9551 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.898803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.899604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.899897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.900180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.913664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.915203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:13.935802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.937061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 297/587] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.942771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.952836 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbd695-494a-4330-8228-b430ecc3a550 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.993321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.994113 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.994617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.994902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.995728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.996048 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.997781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.018086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 291/588] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.018086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.034288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.059594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915490-458c-4044-95ef-f20bc0b0b4e3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.097920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.098593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.098880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.099160 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.099890 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.100200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.101706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.113996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 298/589] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.114862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.132616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.140728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331f48e-8abb-4867-b166-29d33f4a52e0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.179646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.180459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.180826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.181192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.183022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.183606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.185159 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.213703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 292/590] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.213703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.254281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1ce9b-dbd6-46b8-bc6e-d18992e911b8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.294458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.295139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.295428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.295710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.296477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.296794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.298192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.301663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.313443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 299/591] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.314291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.346453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebb29c-0330-4d98-b4a7-ce93a7770ba6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.394905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.395810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.396100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.396386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.412308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=bdccfb4336b7491d8797c3f682ea0687, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.413816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.425548 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719c346-1bd4-42b4-8c04-b7fc699ba5bd None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.436429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 293/592] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:14 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:14.436429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.438407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.486994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.487873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.488229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.488663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.498202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.499820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:14.520052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:14.521183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:14.522249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:14.527017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 300/593] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:14.528248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.577582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61df82-0d07-4bbb-8d1a-9c41adf2756e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.619096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.627281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.628202 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.628600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.628950 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.647257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.649163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:14.674851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:14.675862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:14.676864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:14.676864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:14.676864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:14.676864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:14.676864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 294/594] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:14.677638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.698864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ef107-457f-49ea-9b3e-7f68207d0525 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.749774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.762248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.763062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.763354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.763904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.773608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.776158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:14.796507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:14.797518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:14.798477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 301/595] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:14.799007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.816874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203de21-0837-48ea-a914-febc5f8c60eb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.868175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.868895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.869186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.869580 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.870555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.870882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.872470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.892863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 295/596] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.893530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.930709 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f75569-38d6-4579-ab33-fb38115c2b25 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.948621 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.969869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.970544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.970837 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.971169 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.971862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.972179 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.974561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.992874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 302/597] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.992874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.021967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aee023-5814-4c17-907f-72e12a9afe10 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.038267 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.069213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.070908 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.071699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.072210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.075038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:15.075617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.077360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.094759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 296/598] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:15.095955 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.113082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5dbc45-89e8-46b4-9025-5bc497e8131e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.150314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.151842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.152127 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.152427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.164216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.165807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:15.188359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:15.189425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:15.190383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:15.190383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:15.190383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:15.190383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:15.196003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 303/599] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:15.197139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.230128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.248820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a0e5c-b3cd-49df-b55c-326bb1d27d5e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.302115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.304098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.304874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.305319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.313921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da1e44-742e-423e-8e0c-f71a2d4c331a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.317336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.319100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:15.341139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:15.342422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:15.343583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:15.350905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 297/600] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:15.350905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.362930 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.364005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.364393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.364806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/65534c37a2b8445c8151d802ad980e13/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.382322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=65534c37a2b8445c8151d802ad980e13, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.384072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.400213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.404480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 304/601] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:15 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/65534c37a2b8445c8151d802ad980e13/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:15.406184 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.585220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2708f-2518-40a5-a59c-cdf07e3069d7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.587664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.625688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56f868-4440-4edd-8f28-1087c444360e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.640415 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.641116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.641404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.642047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.642973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:15.643432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.645913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.664860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 298/602] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:15.671337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.685710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.687147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.687451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.688063 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.689137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:15.689498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.691993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.711869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 305/603] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:15.719018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.798038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8f140-b6ff-4e14-9cd7-774a6d3e3180 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.804824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.862613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038b37-3eed-45fb-98db-81655da486e2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.903725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.904688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.904976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.905565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/9f436f3923374b2b97fd95d05bd884f4/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.915627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.924854 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=9f436f3923374b2b97fd95d05bd884f4, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.926441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740ac85-75dc-4b71-aa50-90f9fd806c16 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.929794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.930685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.930977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.931259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.946520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 306/604] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:15 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/9f436f3923374b2b97fd95d05bd884f4/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:15.950131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.951681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.975346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7bc8-c46b-44d1-838f-289f481f9b53 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.975636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7bc8-c46b-44d1-838f-289f481f9b53 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.975920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e7bc8-c46b-44d1-838f-289f481f9b53 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.977387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 307/605] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:15.984024 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.984379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.984728 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:15.998969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:16.000184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:16.001384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:16.007180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 299/606] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:16.010086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:16.016661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:16.017930 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:16.179331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 308/607] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:11:16.187881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b4e2e-0c2b-4872-bb12-001dab59ff40 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:16.211866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:16.237029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:16.237713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:16.238581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:16.238874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:16.239992 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:16.240325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:16.242566 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:16.262145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 300/608] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:16.267737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:16.287549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:16.349498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9040b10-9161-4e70-9b77-4a1349e1efcc None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:16.390242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:16.436065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d4ab3-f459-41b0-a245-2a3a038e6305 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:16.473187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:16.474270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:16.474706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:16.475128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:16.475446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:16.476370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:16.476704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:16.476990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/2aa1864ac1e1481a9c09f418e9978944/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:16.495248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=2aa1864ac1e1481a9c09f418e9978944, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:16.496923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:16.517148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:16.517468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 301/609] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:16 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/2aa1864ac1e1481a9c09f418e9978944/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:16.525262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:16.527196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:16.528297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 309/610] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:16.534490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:16.630143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:16.679030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9eca6-69d6-4b3e-84b8-09e82407e02e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:16.728233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:16.728936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:16.729249 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:16.729529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:16.730303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:16.730625 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:16.732065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:16.741803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 302/611] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:16.747712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:16.889925 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b088b-1129-4f03-9243-7020ae3e95f5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:16.937042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:16.937923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:16.938300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:16.938690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:16.961583 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:16.963744 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-28f7a5a0-e646-4003-9b82-9c93f9d7fb50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:16.991900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:16.994720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:16.997305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:16.997305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:16.997305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:16.997305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:16.997305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 310/612] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:17.000457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.146043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c6ef-da63-492b-901e-d87b56753881 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.146043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c6ef-da63-492b-901e-d87b56753881 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.146043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11c6ef-da63-492b-901e-d87b56753881 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.146043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 311/613] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:17.148885 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.149178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.149458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.164802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e642d-bd5d-47e1-9b7e-b150439c8d17 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.203808 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:17.207876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.208719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.209082 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.209444 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.209602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:17.210339 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:17.210721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:17.210839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:17.212216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:17.227106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 303/614] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:17.237448 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.251536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:17.339581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 312/615] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:11:17.381958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.404457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b485d-d797-46d6-8855-ddfc11debd0d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.482902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.483721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.484009 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.484291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] PATH_INFO: `/v3/users/swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.496677 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:17.498151 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:17.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:17.526535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:17.527768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 304/616] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:17.537659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.568380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccaa681-985d-453f-8c77-759fc107cfd7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.593230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:17.608435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.609282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.609569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.609965 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.623479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:17.632241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:17.633351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:17.634276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 313/617] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:17.643500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.688605 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:17.705438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca0beb-e0fb-4fa4-bc16-8e474a3ba58a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.785394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.786328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.786791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.787222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.788239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:17.788748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:17.790353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:17.825035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 305/618] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:17.830451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.831937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e974b4-05e3-41e1-9848-1f1fc700b82d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.874652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.875638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.876083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.876559 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.877501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:17.878443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:17.880004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:17.890452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 314/619] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:17.902098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.071422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1997-575a-449c-81ec-5df86d867d96 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.083652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.084777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.120954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b4525-0462-4d28-8dae-be35bd36a4ff None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.156614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.158735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.159254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.160365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.169834 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.170786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.171203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.171579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.184291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.185607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.186188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.187427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-3598d9fe-c333-40b7-98b6-0a217f4adafe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:18.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.210475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.211777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:18.211777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:18.211777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.211777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:18.217131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 315/620] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.217131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173a28-739d-4073-8d10-1bc8fd1c5b5f None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.217131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173a28-739d-4073-8d10-1bc8fd1c5b5f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.217131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9173a28-739d-4073-8d10-1bc8fd1c5b5f None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.217131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 316/621] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-627403e3-7113-4b84-9c66-5aabfb773dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:18.223063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.224266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.225265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:18.225265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:18.225265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.225265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:18.227418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 306/622] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:18.227913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9173a28-739d-4073-8d10-1bc8fd1c5b5f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.234877 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.235482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.236007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.256867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:18.258470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:18.358038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 307/623] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:18.358038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.408001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9173a28-739d-4073-8d10-1bc8fd1c5b5f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.457247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.458183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.458750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.459489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:18.460389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.462094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.465163 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.484495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 317/624] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:18.485718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.490370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.516773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717ce81-40fa-485d-a7ff-9126de77917b None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.565475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.566859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.567532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.568051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.569209 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:18.570100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.572010 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.587612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 308/625] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:18.588944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.635845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2bb67b-38b6-4b2c-a085-f9d31079997e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.690129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.690957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.691243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.691522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.706810 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:18.716268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.717432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:18.718352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:18.722898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 318/626] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.724320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.758852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f303f-3582-44a8-9fc6-ee292e55b212 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.794273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.813776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.815885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.816425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.816905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.828280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.829837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:18.844720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.845803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:18.846828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:18.850841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 309/627] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.851800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.865825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e6c09-1f8a-48c5-85db-ff4ccc4e1af5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.871352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.904516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.917610 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.917967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.918246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.930474 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.932944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:18.950173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.951285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:18.952307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:18.956343 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 319/628] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.957105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.982762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ce398-714a-43ab-b47c-140fb5f3c82d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.027410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.028104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.028474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.028757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.029457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.029770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.031324 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471a287-24de-4c2f-9864-ce553966fef8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.040170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 310/629] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.041486 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328cb-d062-4933-8dcd-b7b3e1ea88d2 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.041780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328cb-d062-4933-8dcd-b7b3e1ea88d2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.042105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10328cb-d062-4933-8dcd-b7b3e1ea88d2 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.043506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 311/630] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:19.044003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10328cb-d062-4933-8dcd-b7b3e1ea88d2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.087469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c34715-e09d-49ac-90b2-1b86b475b8e6 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.104885 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.134667 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.135421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.135751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.136088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.137012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.137337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.137480 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.138912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26cabd5-31aa-48b4-a0af-9e1011f95fe5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.157763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 320/631] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.159059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.159358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.159646 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.179151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:19.180383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:19.198233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10328cb-d062-4933-8dcd-b7b3e1ea88d2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.238093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.239077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.239369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.239655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.240502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.240828 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.243035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.264433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 321/632] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:19.264433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.265077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 312/633] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.265077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.371910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081f5b-a9ab-4ced-9255-f0a01d35a9bf None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.373616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.379288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66956dc7-37c2-4e29-9049-d6c467047cdc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.407635 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.409181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.409920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.410209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.410485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.417005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.417656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.417941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.418222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.421316 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.422798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:19.442380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:19.444121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 322/634] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:19.445521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.493920 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.496053 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.552497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c0f5c7-766d-428d-916a-4fcf306c2802 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.589278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.590868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.591297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.591717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/1037a6248c8241828857d69fba1dce7d/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.598454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 313/635] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:19.599325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.609041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=1037a6248c8241828857d69fba1dce7d, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.610513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd69795-482c-43f5-b15b-96c29b653abd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 323/636] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:19 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/1037a6248c8241828857d69fba1dce7d/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e65fd-21d2-459e-89d5-c90dca1d7d01 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e65fd-21d2-459e-89d5-c90dca1d7d01 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e65fd-21d2-459e-89d5-c90dca1d7d01 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 324/637] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:19.632439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e65fd-21d2-459e-89d5-c90dca1d7d01 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.633132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.634945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.729954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd356697-732a-4931-ae10-b7a300d26955 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.762232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e65fd-21d2-459e-89d5-c90dca1d7d01 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.776612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.777563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.777853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.778137 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.790181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:19.798941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:19.800062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:19.801024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 314/638] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:19.802266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.809291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.809986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.810276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.810558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.811307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.811620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.814764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709d5a8-91d9-48f8-8b30-4957ec63931b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.830161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 325/639] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.831367 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.831668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.832019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.855541 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:19.856878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:19.923402 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.924760 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.934530 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 326/640] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:19.935574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.955725 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da9c79-d084-4382-a99b-1f86c14c09ef None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.993000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.993726 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.994030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.994410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.010775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.012336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:20.031059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:20.032063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:20.033040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:20.033040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:20.033040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:20.033040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:20.033040 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 315/641] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:20.033887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.067607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b56de22-2929-49ad-8bbc-fbd4a169edc3 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.106152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.106864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.107152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.107440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.108239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:20.108586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.110037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.119148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 327/642] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:20.119999 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.154133 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34285922-b67b-4787-b5ae-b9c7ca68cb27 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.203833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.208923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.210093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.210502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.210906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.222871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.224368 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.227272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:20.241651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:20.242905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 316/643] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:20.245017 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.259557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23723cb7-2c09-46d6-9b93-c33f7c7b01b6 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.299348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.300460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.300735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.301247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.316379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:20.325037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:20.326122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 328/644] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:20.327116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.372137 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11151188-100b-4b2b-a795-a31cf7859bb2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.415641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.416831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.417354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.417818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.418737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:20.419216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.420920 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.435087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 317/645] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:20.436250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.444606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616bb15-894a-4f71-864c-521352eceb4a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.463201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.486355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.487213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.487518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.487828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.499728 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.501208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.507404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:20.520916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:20.521997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:20.522976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:20.522976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:20.522976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:20.522976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:20.524740 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 329/646] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:20.526579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.563834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e382e-c250-4db0-96c7-41e267f5041c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.601918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.602605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.602899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.603191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.604893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:20.605216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.606666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.630570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 318/647] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:20.631505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.644892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72651d6-1d14-438f-8ef2-466e1fadf033 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.683339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.684331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.684562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.684682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.685456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:20.685763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.687157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.696644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 330/648] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:20.697986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.732572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.761861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc4398-b484-40d8-8de1-5f425d597ee7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.783233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.805157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.806078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.806364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.807787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.811315 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc95a-f6a5-4aa9-8b7a-d0940f1bb7c3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.814923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.816389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:20.830355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:20.831390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 319/649] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:20.832393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.854167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:20.854985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:20.855273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:20.855558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:20.856292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:20.856636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:20.858023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:20.871778 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 331/650] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:20.872664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:20.957583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4fa86c-e979-4960-9600-c0af23ca0ffa None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:20.977290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:20.987873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8996d91-c3ad-4553-9c62-05ce766a3ad8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.007247 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.007929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.008221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.012728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.044584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.045530 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.045896 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.046233 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.058991 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.060483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.069086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:21.078989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:21.079983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 332/651] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:21.080996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.101035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.102809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.193821 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b34de-bc3b-4c52-9c7f-4e6f47d01b2a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.232630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.233295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.233583 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.233970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.234096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 320/652] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:21.234738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:21.235022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.235140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.236502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.249968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 333/653] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:21.250835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.254183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.333087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.361530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dca3d-74b9-45ee-90ad-369504cda978 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.379379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f39ab-3990-4ecf-8ca1-815a93f6772b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.399791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.401296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.401712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.402013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.418788 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.420362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.423869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.424589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.424878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.425160 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.425904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:21.426215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.427610 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:21.436719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:21.437957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:21.439124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 321/654] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:21.443776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.449421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 334/655] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:21.449421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.535629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.562950 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45e663-e472-4917-b977-1c5ad0c99728 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.585466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed34d-8bed-4623-955d-349706de4274 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.601004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.601807 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.602099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.602381 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.614785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.620694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.622268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.636131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.636829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.637122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.637403 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.638252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:21.638575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.640042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.650266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 335/656] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:21.654220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:21.655228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:21.656178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:21.656178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:21.656178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:21.656178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:21.660262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 322/657] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:21.661176 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.661721 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.808879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bc8bb-dde3-42d5-bcb4-fcf110e8d827 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.819264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f1314-d5f8-4b9e-9186-2743d54f8e4e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:21.848478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:21.860704 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.861404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.861695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.861976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.862945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:21.863269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.864898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.878011 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:21.879154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.879449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.879736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.889658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:21.890741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 336/658] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:21.895316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:21.895454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:21.911004 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:21.912085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:11:21.913109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:21.915013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 323/659] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:21.915013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:21.990397 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.038425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a820c54-d8c7-4e86-bb0f-bc0397902802 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.050765 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30152e3-860b-4f01-b211-b53727a7032b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.076051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.076776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.077067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.077351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.078092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.078406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.079912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.094417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.094635 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 337/660] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:22.095454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.095762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.096279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.096783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.148431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.175835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.177336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.217549 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a26565-4d6b-4656-931d-357bd0ced234 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.241470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.261781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.262867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.263432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.263709 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.264600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.264913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.266339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4eaf-3787-47e9-bd0b-6c9281edf296 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.285942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 338/661] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:22.287326 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35a4d8-321f-4976-ba49-0552dd7dd434 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.287619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35a4d8-321f-4976-ba49-0552dd7dd434 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.287901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35a4d8-321f-4976-ba49-0552dd7dd434 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.289529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 339/662] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:22.290091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35a4d8-321f-4976-ba49-0552dd7dd434 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.310037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 324/663] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:22.310037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.414758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35a4d8-321f-4976-ba49-0552dd7dd434 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.416457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.427697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef131d1c-141f-44cb-ba35-7c3dca9ddb2f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.458322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.459140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.459422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.459693 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.467811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.469374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.469526 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.470131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.470418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.470771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ecc705c3-9bcb-4ed5-9796-eac2c44079d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:22.483823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.485006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:22.485984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 340/664] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.486705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d751b-8880-4510-a165-290699d60ffb None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.487052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d751b-8880-4510-a165-290699d60ffb None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.487336 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d751b-8880-4510-a165-290699d60ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.500737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.509369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-796d751b-8880-4510-a165-290699d60ffb None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:22.511556 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-796d751b-8880-4510-a165-290699d60ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:22.560542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.562039 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.600452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 341/665] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:22.600452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d751b-8880-4510-a165-290699d60ffb None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.698360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 325/666] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:22.699643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.705924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.718828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d751b-8880-4510-a165-290699d60ffb None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.756134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.757502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.758489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.765825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.766213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.778328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.779815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:22.798809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.799990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.801185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:22.801185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:22.801185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:22.801185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:22.804629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 342/667] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.805274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.817115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5b344-8377-4e6b-b772-60442931d657 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.861063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.861717 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.862366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.862366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.863067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.863380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.864891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.883954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 326/668] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:22.883954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.925281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d91e5-adcd-4852-a697-dba8f7eb3ab6 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.965754 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.966473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.966763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.967029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.978779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:22.986863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.988101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:22.989239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:22.992955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 343/669] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.993870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.998106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.010728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc3f53-159d-4415-bba6-5387f68bfcb5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.022064 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.053869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.054682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.054975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.055259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.067735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.069244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:23.089713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.090918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.092069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:23.092069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:23.092069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.092069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:23.095480 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 327/670] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.096334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.116019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4936eb-1a24-4138-8923-bfab51c7161f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.154114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.154790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.155081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.155368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.156090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.156434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.157814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.175147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 344/671] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.175147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.216614 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa70c16-1449-40b5-abab-20cebc8c64b6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.255154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.255402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.256540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.256913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.257264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.258286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=65534c37a2b8445c8151d802ad980e13) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.258688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.260789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.268671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 328/672] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/role_assignments?user.id=65534c37a2b8445c8151d802ad980e13&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.269693 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.303144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc704c5d-7543-4226-a275-b4e448837640 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.316065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.350537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.351491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.351921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.352309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.353268 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.354248 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.355785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.365892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 345/673] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.367026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.430023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e76280-6bf2-4268-8a8d-c614914c47fd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.485115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.485831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.486478 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.486841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.488198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.488635 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.490706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.500681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0f44d-fa25-4077-8d18-f24e2f1a12f4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.510798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 329/674] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.511749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.546796 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.568814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.569630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.569921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.570201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.582763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.584876 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:23.610098 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.611158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:23.612099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 346/675] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.613190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.648500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.669757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911f0ed-dc99-459d-b139-976b04d615bb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.739671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.740428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.740884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.741175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.755791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dab7e-5149-4bbe-a1ae-3cfa4ab036f0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.761939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.763433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:23.782195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.783222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.784155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:23.784155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:23.784155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.784155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:23.788052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 330/676] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.788942 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.799290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.800455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.800975 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.801460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.810954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.812847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:23.827979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.829018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.830037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:23.837107 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 347/677] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.838485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.891246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.901600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb61f0-7efe-42a3-9745-e6b1ff9b1e67 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.944165 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.944897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.945186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.945466 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.946797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.946797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.948902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.956572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.962596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9e45-78fc-4dac-b261-1414a570de3c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.965163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 331/678] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.966199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.005383 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.006394 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.006905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.007399 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.008351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.008908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.011226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.025496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 348/679] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:24.026835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.085264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50815-111c-4ed3-8694-c8373d7474a2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.122400 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.123089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.123377 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.123655 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.124803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.125122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.126598 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.139701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 332/680] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:24.140904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.144730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0092f-af83-4751-95a7-f85e514404ef None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.154135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:24.186615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.187224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.187504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.187777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.188636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=9f436f3923374b2b97fd95d05bd884f4) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.188951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.190412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd274c-c48a-4598-a58e-1ef4f937e5ec None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.204830 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 349/681] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/role_assignments?user.id=9f436f3923374b2b97fd95d05bd884f4&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:24.207783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6266-2d7a-4b34-ac51-7859ea2ebac7 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.208529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6266-2d7a-4b34-ac51-7859ea2ebac7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.209138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c6266-2d7a-4b34-ac51-7859ea2ebac7 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.211459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 350/682] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:24.212555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c6266-2d7a-4b34-ac51-7859ea2ebac7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.233241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:24.301491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c5d74-e680-45f4-9754-b0d5bacb9429 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.347068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.348089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.348737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.349007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] PATH_INFO: `/v3/users/swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.358800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.360527 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.372293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c6266-2d7a-4b34-ac51-7859ea2ebac7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5c35c605-0ee7-49ec-afa4-3a9ceb91eccc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:24.378407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:24.380079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 333/683] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.382130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.383457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.404674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:24.406044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:24.421393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.422385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.422875 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.423161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.423999 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=2aa1864ac1e1481a9c09f418e9978944) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.424329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.425801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.433404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 351/684] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/role_assignments?user.id=2aa1864ac1e1481a9c09f418e9978944&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:24.434113 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.486772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 334/685] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:24.510931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.515450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:24.589878 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:24.668934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b36a42-011e-49be-b731-c942e61b0ee1 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.713884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.714617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.714910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.715194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.716054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.719517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.719517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.726918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d4cd-0c8f-49b4-9334-393665abb196 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.747774 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 352/686] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:24.751833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.789534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.791209 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.792358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.793490 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.812665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:24.820799 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:24.821865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:24.823341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 335/687] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:24.828667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.875962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:24.891048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c56c5-e472-4fe6-9e25-5f46fd7c5faa None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.943041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.944062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.944580 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.945068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.984903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53426e8e-be8f-411c-b824-eccd6181dcc9 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.997255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:25.044731 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.046438 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.053250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.053967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.054261 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.054544 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.055344 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:25.055668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.058072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.082759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 336/688] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:25.087721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.173014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:25.190640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 353/689] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:25.192237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.262207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b057c9-167b-4ca8-af39-a91587460c29 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:25.302705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.303525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.303817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.304103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.324817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.326334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:25.352708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:25.354291 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:25.355266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:25.355266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:25.355266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:25.355266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:25.365689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 337/690] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:25.371930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.391738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c014b4a-2d11-49bf-8ca2-63d9c9331896 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:25.424972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:25.487117 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.488563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.488861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.489143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.516883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34ab5b-2213-4c57-b3b0-4d6c1ab18406 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:25.525978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.527645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.556013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.557510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.557810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.558100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.558853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:25.559173 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.560620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:25.573006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:25.574016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:25.575121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:25.575121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:25.575121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:25.575121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:25.575372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:25.578909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 354/692] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:25.584883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.586287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.691507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:25.752819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:25.786075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e007c8c-f38d-4e16-9dc3-c01a8453138c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:25.790539 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50ce5-e99e-4cc3-9454-6fd5a4ffa161 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:25.860214 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.861090 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.861384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.861671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.864640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:25.865347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:25.865637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:25.865920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:25.866691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:25.867036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.868608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.878969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:25.880506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:25.887744 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 355/693] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:25.898502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:25.927930 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:25.929046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:25.930590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 339/694] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:25.936689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.090825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b66c07-ad07-4b3e-aff0-37430708992f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.093465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.124168 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ad3ac-a5f8-40fb-86ea-0593ffb35c43 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.145789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.146595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.147335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.147616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.158136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.159977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.162861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.163569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.163857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.164147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.165066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.165385 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.167395 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcc89a-1e52-4062-ae7e-0b75710b570d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-270d306b-1041-44cd-9439-d9b63a18a1b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:26.178868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:26.179841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:26.181928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:26.183573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.188419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 356/695] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:26.188419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d456b-2b67-4a9e-bd55-2fcd70849b82 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.188419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d456b-2b67-4a9e-bd55-2fcd70849b82 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.188419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d456b-2b67-4a9e-bd55-2fcd70849b82 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.194704 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 357/696] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:26.194704 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d456b-2b67-4a9e-bd55-2fcd70849b82 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.195573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 340/697] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:26.200463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.200463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.200463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.223528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:26.225157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:26.309603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 341/698] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:26.310875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.366020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d456b-2b67-4a9e-bd55-2fcd70849b82 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.407391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.408067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.408356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.408668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.409475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.409781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.411262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.430557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 358/699] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:26.430557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.457298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.485449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206510b-26a9-4c8d-a6f1-8446c7f6423e None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.526351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.535446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.536707 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.537679 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.538331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.558108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bf827-008a-4570-9400-ca952a800930 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.598392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.599291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.599941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.599941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.613790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.615403 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.636742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.638545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:26.644492 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:26.645479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:26.646460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:26.646460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:26.646460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:26.646460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:26.652023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 359/700] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:26.653130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.768316 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.789721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 342/701] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:26.789721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.804575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87281ba-df80-4b60-9a32-b727a5b4db3b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.831877 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.856048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.856922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.857145 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.857438 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.858298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=1037a6248c8241828857d69fba1dce7d) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.858605 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.860078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.867617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 360/702] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/role_assignments?user.id=1037a6248c8241828857d69fba1dce7d&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:26.868569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.937572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac93a2c-18cc-4dda-b1fc-4ae099431a44 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.993077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.994759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.995864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.996069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.996873 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.997196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.998641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de02f8a-74d7-40d6-b1f0-94bdcd9e7062 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.012892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b27f-aa1d-41bb-889f-240f2b35fad5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.017092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 343/703] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:27.018595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e679b-c32b-4434-afbf-10e01b4a37d2 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.018893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e679b-c32b-4434-afbf-10e01b4a37d2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.019187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e679b-c32b-4434-afbf-10e01b4a37d2 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.021617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 344/704] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:27.022503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e679b-c32b-4434-afbf-10e01b4a37d2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.064016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.064929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.065220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.065502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.077020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.080771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.085038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-f416b792-bf1a-4c39-b06b-7ac27b2a888b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:27.110514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:27.111733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:27.112849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:27.112849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:27.112849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:27.112849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:27.113793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 361/705] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:27.115178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.115476 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.115761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.138735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:27.140050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:27.164192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.178503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e679b-c32b-4434-afbf-10e01b4a37d2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.228008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.228894 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.229180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.229459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.230295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.231957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd9513-9b66-4a12-9993-8d34d39bc05d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.238787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 362/706] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:27.239692 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.259114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 345/707] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:11:27 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:27.261430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975417-3af3-4f35-807b-716221944fe4 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.261728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975417-3af3-4f35-807b-716221944fe4 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.262015 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04975417-3af3-4f35-807b-716221944fe4 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.263964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 346/708] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:27.274152 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.274554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.274956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.296594 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:27.297929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:27.371663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36c08d-b89a-4c12-9d88-d7c0290b7e51 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.392428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 347/709] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:11:27.392428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.416428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.436608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.437628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.438144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.438613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.439565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:27.440095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.441772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.456656 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 363/710] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:27.457872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.487497 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.519148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626cb98-8b94-476d-9ff6-98c8d387c140 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.596252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4066500-a6c1-46c0-ab4d-c56c1772e7d8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.639109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.639939 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.640232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.640549 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] PATH_INFO: `/v3/services/network` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.641231 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.642638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:27.658185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:27.659443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:27.661285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 364/711] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:27.662338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.694609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.695534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.695891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.696431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.697883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.699370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:27.715905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:27.717094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:27.718201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 348/712] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:27.719150 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.725290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.796646 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af495ea2-f10c-4aa5-9101-4042c06b2999 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.837123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.838575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.838867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.839152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.847456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.849106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.852842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81155b5-cc78-4a2d-8819-d4db106668ce None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:27.865353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:27.866473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:27.867594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 365/713] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:27.869568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.890864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:27.891541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:27.891825 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:27.892100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:27.892850 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] RBAC: Adding query filter params (name=network) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:27.893156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:27.894540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:27.899619 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 349/714] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:27.901079 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:27.904986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:27.989557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef2439-3b64-471b-9230-c95929fc4d62 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:27.998431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:28.012153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70cf8a-53d9-460a-bbb9-a4bcdc6e538b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:28.033202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:28.034005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:28.034300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:28.034584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:28.035389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:28.035705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:28.037322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:28.043196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 366/715] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:28.044181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:28.049359 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:28.050036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:28.050326 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:28.050607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:28.051417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:28.051732 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:28.053219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:28.067179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 350/716] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:28.068169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:28.141797 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:28.163400 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a869a-371f-4061-885a-003a82376b8d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:28.183716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea512c-9982-4c1c-9493-f6846c2f23a5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:28.214003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:28.214738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:28.215033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:28.215314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:28.216164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] RBAC: Adding query filter params (type=network) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:28.216593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:28.217985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:28.221179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:28.221853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:28.222139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:28.222416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:28.223140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:28.223447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:28.224572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 367/717] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:28.224965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7b6df-401c-4ff0-9149-a932b4c32299 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:28.225430 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:28.232541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 351/718] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:28.270632 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:28.355158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7724f7-153d-4218-94bc-6421ebca4e49 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:28.397115 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:28.397812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:28.398103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:28.398416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:28.399238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:28.399564 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:28.401841 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b713ad-5be9-4aa9-9b44-1d0cc69f8621 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:28.408939 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 368/719] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:28 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:29.720932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d86c16-500e-46b8-9c1d-9215ba79ab03 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:29.721507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d86c16-500e-46b8-9c1d-9215ba79ab03 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:29.721507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d86c16-500e-46b8-9c1d-9215ba79ab03 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:29.723158 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 352/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:29.729335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:29.729860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:29.729941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:29.754370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:29.755613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:29.874532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 369/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:29.910823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d86c16-500e-46b8-9c1d-9215ba79ab03 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.074404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d86c16-500e-46b8-9c1d-9215ba79ab03 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:30.114981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:30.128025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:30.128890 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.129180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.129463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.148368 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-981194d8-024e-4154-a574-57a73bfc8193 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:30.157606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:30.158699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:30.159622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 353/722] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:30.165337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.179993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337719-b584-4bcf-affc-d51971a2b4e1 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.180323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337719-b584-4bcf-affc-d51971a2b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.180671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d337719-b584-4bcf-affc-d51971a2b4e1 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.182653 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 354/723] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:30.188459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.188753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.189041 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.209765 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:30.210988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:30.227559 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:30.321576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 355/724] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:30.348805 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.373388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fd50b-4a7d-42e3-b511-5b0729f8af91 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:30.443280 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:30.443969 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.444254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.444604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.445371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:30.445682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:30.447576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:30.470052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 370/725] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:30.475833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.550590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89efb6f-362b-4400-91d0-ab4affcbb5e1 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:30.586005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:30.613759 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:30.623675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf49f1-af46-4b8d-9d9f-231b9d57eb4c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:30.626076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:30.627606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.628288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.629089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.662519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:30.663463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.663754 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.664029 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.664737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:30.680433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:30.680433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:30.684231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:30.685419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:30.686369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:30.689337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:30.697733 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:30.698633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:30.699597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:30.700602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:30.700602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:30.700602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:30.700602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:30.700602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 371/727] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:30.706386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.873431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb70579d-3769-48d6-b61b-5247ce5e2284 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:30.913719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:30.914755 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:30.915143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:30.915528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:30.917029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:30.917459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:30.920453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:30.933722 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 357/728] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:30.937387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:30.989133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb838e-d612-48ee-8ff3-5232dc0486c6 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.028990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.088274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.089128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.089417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.089695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.090439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:31.090753 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.092160 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcdc075-64cb-4865-942c-b76abe20a2da None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.096695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411c555-3838-48ec-be88-eee03419fac7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.126866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 372/729] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:31.134086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4ace-798f-482d-abc6-9fd151865441 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.134404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4ace-798f-482d-abc6-9fd151865441 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.135182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab4ace-798f-482d-abc6-9fd151865441 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.136968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 373/730] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:31.137244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.137765 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4ace-798f-482d-abc6-9fd151865441 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.138072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.138361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.138643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.140722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.155228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.156834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-7f0b08ff-aacb-4037-b3e4-b2142af3a79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:31.185791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:31.186821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:31.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:31.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:31.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:31.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:31.187769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 358/731] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:31.192910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.192910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.192910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.210061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:31.211757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:31.266469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab4ace-798f-482d-abc6-9fd151865441 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.300558 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 359/732] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:31.300558 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.313288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.314370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.314817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.315298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.323455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.324977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.333203 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-3a0b92b2-4f7d-45ca-816e-ef98617650a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:31.340835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:31.342196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:31.343212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 374/733] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:31.343839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9fa-c317-4dda-aa21-60a6abff3bd9 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.343839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9fa-c317-4dda-aa21-60a6abff3bd9 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.343839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82a9fa-c317-4dda-aa21-60a6abff3bd9 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.345299 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 375/734] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:31.349277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82a9fa-c317-4dda-aa21-60a6abff3bd9 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.419929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887c58a-2ec4-4b55-96f8-416e5cac3356 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.421903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.457203 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.457879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.458169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.458458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.459865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:31.460463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.462462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.463420 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82a9fa-c317-4dda-aa21-60a6abff3bd9 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.477537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 360/735] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:31.478889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.504693 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.505369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.505661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.505940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.506668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.508054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa6e2c-a8ac-4b3b-908d-291014172160 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.523761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 376/736] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:11:31 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:31.525817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f3cfa-1274-45e1-9eb8-83bddb908062 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.526329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f3cfa-1274-45e1-9eb8-83bddb908062 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.526628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f3cfa-1274-45e1-9eb8-83bddb908062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.545484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-074f3cfa-1274-45e1-9eb8-83bddb908062 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:31.546759 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-074f3cfa-1274-45e1-9eb8-83bddb908062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:31.570498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.594990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18131a-1ab5-4b61-80e0-97143f4feaf7 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.642848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.643536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.643993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.644141 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.644998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:31.645315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.646801 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12fc08-1c1c-4c23-9c26-742c8317ef00 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.659992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 377/737] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:31.659992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f05a-73b8-4049-a7d8-f1f0c41fd019 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.659992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f05a-73b8-4049-a7d8-f1f0c41fd019 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.659992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3f05a-73b8-4049-a7d8-f1f0c41fd019 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.660870 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 378/738] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:31.663655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3f05a-73b8-4049-a7d8-f1f0c41fd019 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.676493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 361/739] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:31.677815 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.678212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.678502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.699372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:31.700810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:31.733101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.794125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3f05a-73b8-4049-a7d8-f1f0c41fd019 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.798628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 362/740] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:11:31.799142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.834616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.835252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.835540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.835820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.845408 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:31.846869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:31.862540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:31.863613 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:31.865157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 379/741] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:31.865814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:31.888457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:31.931215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0b3d0-0239-4d18-ac2e-e5a1e5406a63 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.968187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:31.968993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.969268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.969547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.979363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a4560-a153-4026-b815-c06bda20b0b4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:31.989820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.018863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.019660 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.019943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.020218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.032367 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:32.040269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:32.042020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 380/742] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:32.042979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.052962 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.054464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.136660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.162214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a540bb-de6c-4551-88f4-32e106c539fc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.176694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 363/743] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:11:32.177624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.214395 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.215078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.215369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.215652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.216498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.217813 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.232560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 381/744] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:11:32 2022] POST /identity/v3/services => generated 240 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:32.233785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.277848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.326305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e00fae-f12f-4299-9786-524b034ac03e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.380577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.381396 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.381748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.381977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe07cf-f642-49ca-88cf-949b4f812103 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.382154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.383187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:32.383573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.386925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.419474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 364/745] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:32.419474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.431361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.432748 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.433236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.433705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.434772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:32.435306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.437011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.444531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.449256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 382/746] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:32.450727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.550063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8eb65-340d-403d-9153-fb1fd6b90109 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.579690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.598011 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.598219 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9eddd-b1d4-4469-bd9e-c9322d479fcc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.598874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.599218 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.599508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.611656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.613227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:32.633411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:32.634432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:32.635439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:32.635439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:32.635439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:32.635439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:32.639739 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 365/747] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:32.640641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.662343 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.663469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.663937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.664444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.761606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.771761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.791204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.843701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c723fa-97dc-4acf-8103-c972bd46fac2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.877908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.885793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.886791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.887081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.887410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.895916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 383/748] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:32.897369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.898763 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.900645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:32.926703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:32.927761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:32.929229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:32.929229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:32.929229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:32.929229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:32.932710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 366/749] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:32.933823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.017573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34020c93-45e8-41d2-ab5b-d8761600f7f4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.049440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e123-23e5-4828-b506-260d2dbc7392 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.068235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.069305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.069801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.070309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.071296 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.071910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.074073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.080269 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.090727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 384/750] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:33.091981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.092758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.093575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.093987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.094263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.106063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.107691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:33.127744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:33.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:33.129782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:33.129782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:33.129782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:33.129782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:33.129782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 367/751] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:33.130545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.151658 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.215038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dbb5a-427b-4c0f-ab00-0a9499b1b420 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.250207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f6688-1f6a-4731-a910-471555f66699 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.254565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.255486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.255980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.256517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.257479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.258013 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.259879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.274672 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 385/752] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:33.276540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.295244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.296102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.296560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.296867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.306275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.308730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:33.326842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:33.327865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 368/753] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:33.328904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.379573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.396372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb98ea5-99db-4dda-a846-da5a248ab235 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.408818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.442509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.443589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.444111 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.444662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.446417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.448020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.450562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.459113 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eca093-c6dc-4012-939a-a6a549209bf2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.468243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 386/754] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:33.469962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.499719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.500649 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.500934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.501210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.520438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.523447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:33.549201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:33.550988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:33.552853 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:33.556058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 369/755] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:33.558545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.620766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7de90-3fe7-4634-b876-7824d460851a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.664475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.665161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.665520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.666350 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.667167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.667486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.669122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.684698 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 387/756] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:33.685576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.692271 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.701589 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de959c-12dd-4a5d-932f-e7f9010d682b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.723459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:33.756315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.757727 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.759214 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.760029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.774404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.777820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:33.802171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:33.803213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:33.804193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:33.812307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 370/757] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:33.812307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.826398 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2355-a03a-4519-9457-3a654b8bff5f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.881143 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.882677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.883489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.884456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.885889 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.886683 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.889142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:33.913624 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 388/758] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:33.914913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:33.942642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5499-3424-49f9-ab19-a7e3828a751d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:33.982462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc95024-af32-4951-809c-1038e685093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:33.983127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc95024-af32-4951-809c-1038e685093a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:33.983414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc95024-af32-4951-809c-1038e685093a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:33.983795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc95024-af32-4951-809c-1038e685093a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:33.984673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc95024-af32-4951-809c-1038e685093a None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=1037a6248c8241828857d69fba1dce7d) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:33.984990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc95024-af32-4951-809c-1038e685093a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:33.986568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc95024-af32-4951-809c-1038e685093a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.000435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 371/759] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/role_assignments?user.id=1037a6248c8241828857d69fba1dce7d&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.000435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc95024-af32-4951-809c-1038e685093a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.020620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.044105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.065599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c254a4e-d87c-4249-a7ba-300280ff8594 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.107840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.108592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.108885 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.109168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.110140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:34.110338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.111879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98becf-9670-4656-8365-4a8c644b9231 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.140001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 389/760] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.140001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be230-d5d4-4e28-a2ba-4d72ace7a584 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.140001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be230-d5d4-4e28-a2ba-4d72ace7a584 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.140001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8be230-d5d4-4e28-a2ba-4d72ace7a584 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.140001 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 390/761] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:34.141441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be230-d5d4-4e28-a2ba-4d72ace7a584 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.144881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc95024-af32-4951-809c-1038e685093a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.200156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.201268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.201833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.202294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.215378 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.216943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b14efe5c-cf67-4ad8-a207-3478c10b69a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:34.237982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:34.239045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:34.240127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:34.240127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:34.240127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:34.240127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:34.240127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 372/762] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:34.244453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.244453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.244453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.265188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:34.266833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:34.278549 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8be230-d5d4-4e28-a2ba-4d72ace7a584 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.319322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.319513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.319806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.320799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=4daedcbeedb148b19078761a34b8d4a1) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:34.321162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.323842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.331053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 391/763] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/role_assignments?user.id=4daedcbeedb148b19078761a34b8d4a1&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.332028 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.352251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 373/764] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:34.361293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.380329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.383298 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.464456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7edbcb-981c-4507-b44c-f1b8fc8097a3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.503980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad27ce-4bf7-4443-8a8e-853179afb0e8 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.508489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.509178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.509498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.509790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.511282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:34.511600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.513060 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.527625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 392/765] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.533479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.551351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.552752 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.553319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.553836 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.554773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.556931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:34.571193 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:34.572419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:34.573414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:34.576906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 374/766] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:34.583366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.632840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.655710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf072572-f833-4926-b386-bdeaa8b24e56 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.659144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.699677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.700587 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.700880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.701164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.702712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10ec3b-404a-46ab-81b4-d793038db51a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.709060 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.710533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:34.726156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:34.727203 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:34.728183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 393/767] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:34.733547 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.743440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.745337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.745909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.746428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.747423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:34.747977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.749686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.755468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 375/768] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.762053 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.856908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4aed8-ef77-49c2-881a-022b8c4b6c0f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:34.894194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:34.900542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:34.901988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.902463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.902897 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.904273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:34.904802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:34.906976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:34.921743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 394/769] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:34.927583 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:34.963972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944c2c-f2b6-4b1d-a100-dfcab32f5448 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.048872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.062138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.064745 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.065292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.065910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.066879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:35.067601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.069464 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.076344 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 376/770] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:35.082529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.112192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98cb400-ffec-420f-95b1-79625c1ca088 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.148059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.149118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.149405 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.149682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.152783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=bdccfb4336b7491d8797c3f682ea0687, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.154604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594013a0-03b6-4cbd-93f6-fdb450b25861 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.172568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 395/771] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:34 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:11:35.174099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eccb0-2e79-478b-8c44-f0715b605bfc None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.174443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eccb0-2e79-478b-8c44-f0715b605bfc None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.174727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774eccb0-2e79-478b-8c44-f0715b605bfc None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.176699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 396/772] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:35.183126 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.184346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.184806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.209862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:35.211558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:35.217565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5355a7a-161f-45f1-9813-bb5ff9bd198f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.258459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.259516 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.260822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.261391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.262537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] RBAC: Adding query filter params (interface=public, service_id=2e600eb63aec450ab15cb7973a54095e, region_id=RegionOne) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:35.263087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.264904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.274571 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 377/773] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/endpoints?service_id=2e600eb63aec450ab15cb7973a54095e&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:35.280384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.281282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.329366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 397/774] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:35.387005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.403065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.444192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0915c-633a-42a7-b10e-2bde867014bb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.497895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.499031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.499539 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.500059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.501073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.502753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.511361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 378/775] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:35.697874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dc9dd-39b0-4cef-8f33-468fd75812cb None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.747437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.748277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.748601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.748880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] PATH_INFO: `/v3/services/network` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.749582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.751049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:35.763864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:35.765072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:35.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 398/776] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/services/network => generated 88 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:35.773680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.899896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43632-c3ed-44b9-8c2a-9a0cf3f3887d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.926514 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.942089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.943120 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.943534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.943937 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.945321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] RBAC: Adding query filter params (name=network) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:35.946005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.948889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.955111 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 379/777] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:35.960682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:36.053311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:36.115517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b09484-48fd-4e00-95a1-f5e3c8ba1759 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:36.153269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:36.153968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:36.154262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:36.154550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:36.155307 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] RBAC: Adding query filter params (type=network) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:36.155623 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:36.157274 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:36.163486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 399/778] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:36.170232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:36.328731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c91071-f44e-4611-b4f7-75da053cb76b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:36.374776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:36.375449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:36.375736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:36.376019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:36.376870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] RBAC: Adding query filter params (interface=public, service_id=7ba144f6609e489bada26e5add44f21d, region_id=RegionOne) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:36.378901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:36.380311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fff3e-ac19-48bc-96e0-396338377b0d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:36.386575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 380/779] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:11:36 2022] GET /identity/v3/endpoints?service_id=7ba144f6609e489bada26e5add44f21d&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:36.392538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:36.587520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3544bbc-5709-45f5-b421-b908fa30d435 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:36.604387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:36.641380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:36.642503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:36.643036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:36.643557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:36.644546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:36.646263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc899c6-dec0-464d-908c-db051323c3b8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:36.653934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 400/780] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:11:36 2022] GET /identity/v3/services => generated 772 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:36.939174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5fdbe-0372-4c8b-a7f3-487cabfebb39 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:36.939587 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5fdbe-0372-4c8b-a7f3-487cabfebb39 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:36.939776 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5fdbe-0372-4c8b-a7f3-487cabfebb39 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:36.941462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 381/781] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:36.947244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100be6-c68d-49b5-a0b9-dc897f797711 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:36.947829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100be6-c68d-49b5-a0b9-dc897f797711 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:36.948355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100be6-c68d-49b5-a0b9-dc897f797711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:36.968833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-c8100be6-c68d-49b5-a0b9-dc897f797711 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:36.970424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-c8100be6-c68d-49b5-a0b9-dc897f797711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:37.053978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 401/782] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:37.078056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5fdbe-0372-4c8b-a7f3-487cabfebb39 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.090079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01872eea-f5ba-413a-a539-35b364026798 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.090750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01872eea-f5ba-413a-a539-35b364026798 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.091267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01872eea-f5ba-413a-a539-35b364026798 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.092942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 402/783] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:37.099254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.099864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.100362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.128269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:37.133124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:37.188813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:37.240281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 403/784] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:37.286531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5fdbe-0372-4c8b-a7f3-487cabfebb39 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:37.286785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.330736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:37.331544 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.331829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.332107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.347595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:37.357437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:37.358555 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:37.359483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:37.364881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 382/785] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:37.365023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:37.373288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.515206 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4862b29-dec4-4314-ac5a-c56b80214fe7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:37.515411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17dc6-22c0-4274-baf5-5c0ed95060c5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:37.547929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:37.555212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:37.555928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.556218 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.560743 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.561600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:37.561917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:37.563330 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:37.582496 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 383/786] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:37.582696 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:37.583311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.583597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.583874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.589485 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.613431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:37.628485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:37.629663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:37.630573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 404/787] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:37.637582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.796008 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:37.805133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46130bf7-fd39-4e9d-8e06-f5947eaea27f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:37.839469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d2182-8a29-47e3-bc31-d29740d7efea None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:37.847207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:37.847892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.848175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.848473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.849230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:37.849542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:37.851444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:37.862148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 405/788] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:37.867858 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:37.914818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:37.915671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:37.915966 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:37.916252 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:37.942702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:37.944196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:37.970749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:37.973475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:37.975709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:37.975709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:37.975709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:37.975709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:37.975709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 384/789] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:37.983765 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.049094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.077121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dc26f-9dc0-4da6-8660-e7744769f969 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.144553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.145366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.145629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.145908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.167919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.169409 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:38.188070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:38.190374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:38.192732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:38.192732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:38.192732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:38.192732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:38.193269 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca1f79-de75-4d4d-81d2-a3e0930bb27e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.196989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 406/790] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:38.196989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.207781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.251533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.252685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.253191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.253701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.255286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:38.256081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.257880 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672a7f-e173-4a93-a7d5-de757c04e588 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.277392 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 385/791] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:38.279100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f272d2-ef8e-448b-9aac-bc4f3f615817 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.280225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f272d2-ef8e-448b-9aac-bc4f3f615817 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.280832 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f272d2-ef8e-448b-9aac-bc4f3f615817 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.283126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 386/792] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:38.284651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f272d2-ef8e-448b-9aac-bc4f3f615817 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.316389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb33d3-0e52-4557-a2c9-e8d66f20b473 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.359980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.360804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.361091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.361531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.362292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:38.362654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.364091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178eae2e-c04c-4866-b06d-5d767497034a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.378625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 407/793] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:38.379962 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.380257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.380570 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.396119 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.402517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:38.404105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:38.423248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f272d2-ef8e-448b-9aac-bc4f3f615817 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.474694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.475830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.476124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.476433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] PATH_INFO: `/v3/users/swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.477327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.485177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 408/794] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:38.486236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.488120 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.490862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:38.511991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:38.513352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:38.514508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 387/795] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:38.518822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.608900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050c82e-ca5c-4d99-9f1e-18bf42f8557f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.639189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98be9c5-abb7-4b05-96af-249b5363186f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.662360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.663189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.663472 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.663752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.672720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.674181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.689035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.689721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:38.689864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:38.690924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:38.691952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:38.692576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.692576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.692576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:38.692576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.694686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcdd76-2521-47a8-9c49-103ad1f976cb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.695832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 409/796] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:38.696715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.715364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 388/797] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:38.716860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab3c1-8340-45a7-9b23-0d9b5e88da92 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.717153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab3c1-8340-45a7-9b23-0d9b5e88da92 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.717433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab3c1-8340-45a7-9b23-0d9b5e88da92 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.723161 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 389/798] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:38.723949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab3c1-8340-45a7-9b23-0d9b5e88da92 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.726957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.768357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:38.818711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7cd96-123c-46ed-9ea2-70970576e54b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.845392 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab3c1-8340-45a7-9b23-0d9b5e88da92 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:38.857304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.858111 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.858473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.858986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.875840 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:38.883622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:38.884711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:38.885628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 410/799] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:38.886390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:38.891430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:38.892132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.892445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.892730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.893595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:38.893909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:38.895353 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc87c6-6665-43a1-a89e-f14b5f462430 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:38.919524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 390/800] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:38.919524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:38.919524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:38.919524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:38.940172 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:38.941488 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:39.001673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.009002 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3878d-7884-4b49-974f-46a982030491 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.020804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 391/801] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:39.020804 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.039818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.060781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.061602 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.061878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.062158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.074607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.076081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:39.094328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:39.095335 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 411/802] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:39.096327 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.154634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e53c2-67f3-4463-85c1-fa797bf7ac3f None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.207424 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.209191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.209651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.210364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.211460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:39.211933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.214690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.218575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56def-859e-4f60-8604-c92253f94ce1 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.227629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 392/803] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:39.230259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.257954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.258953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.259388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.259734 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.271386 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.272872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:39.294987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:39.296214 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 412/804] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:39.297205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.303218 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.334920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.374055 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2773f7-16ec-4bd3-802f-101bdaac64db None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.415929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.416701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.416996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.417281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.417960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.419360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.421052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bcc73-8a87-4e03-81ec-0b4394d2dbeb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:39.432974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:39.434138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:39.435037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 393/805] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:39.436532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.467614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.468668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.469154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.469633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.470552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:39.471046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.472648 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.486226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 413/806] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:39.487518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.554927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28ca52-5e60-4ce4-afcb-57fa3c832ebf None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.572466 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.575016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.591512 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.592340 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.592656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.592935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.601522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1063b89-543f-455f-86a6-9e3059e6e75b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.604848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.606334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:39.624307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:39.625347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:39.626329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:39.626329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:39.626329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:39.626329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:39.626329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 394/807] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:39.626644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.643903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.643903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.643903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.643903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.644538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:39.644538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.645890 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.658487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 414/808] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:39.659940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.744914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78704db0-afe3-4f79-865d-3350658e23c8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.774535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14bab9-95f0-4b06-ae68-53cdf23057a0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.782168 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.782844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.783137 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.783423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.784149 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:39.784500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.785931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94721db1-bb54-4205-a2f7-a314c9ac21ac None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.790479 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 395/809] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:39.792521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca18-505f-4e7c-a240-ca3a5871a0ee None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.792819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca18-505f-4e7c-a240-ca3a5871a0ee None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.793107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6ca18-505f-4e7c-a240-ca3a5871a0ee None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.794370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 396/810] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:39.802867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ca18-505f-4e7c-a240-ca3a5871a0ee None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.816878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.817687 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.817976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.818252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.826102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.827556 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.837035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:39.841982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:39.843110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 415/811] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:39.844363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:39.847468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:39.914323 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6ca18-505f-4e7c-a240-ca3a5871a0ee None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.950028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:39.950827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:39.951128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:39.951421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:39.952196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:39.952542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:39.953997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:39.964166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40def0c0-8477-4edc-b129-717c6d80e425 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:39.969441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 397/812] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:39.971109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.000904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.001624 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.001942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.002222 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.018974 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.020803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-7a5d6e2b-b24f-45af-a908-2414c6da3dc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:40.036751 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:40.038703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 416/813] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.040082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.041629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.058901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:40.060148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:40.078060 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.088315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad390b-c987-44af-979f-094cee6c5ffa None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.120360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.131558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 417/814] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:40.132431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.143439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.144127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.144442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.144729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.145450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:40.145767 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.147156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.156570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 398/815] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:11:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:40.157942 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.244644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f752e72-96cd-4a89-8740-5be13f6b973d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.283463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.283859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398345dc-ee10-44a6-89ab-f5d21d690cb6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.284309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.284309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.284620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.285776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:40.286091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.287550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.299775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 418/816] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:40.300647 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.334710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.335509 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.335795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.336073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.344029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.345518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.357238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:40.360032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:40.361212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:11:40.362457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:40.365809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 399/817] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:40.366678 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.385692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.406948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d527758-2d81-4b56-b008-c4feecc1af44 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.443958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.444658 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.444940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.445217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.446000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:40.446302 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.448290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.460805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 419/818] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:40.461642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.472994 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d305e51-96e2-43e0-9a88-a9db00cd0991 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.511827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.513029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.513316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.513581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] PATH_INFO: `/v3/services/network` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.514250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.515647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:40.527830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:40.534403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:40.535531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 400/819] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:40.536611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.571259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baf91d-3b1d-4f13-b2c0-d6afd7fb4064 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.586903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.610105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.610812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.611233 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.611644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.612523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.613919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.629210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.639795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 420/820] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:11:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:40.640371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.655662 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c51d9-9c60-42ad-9f90-7efa37a38e7f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.713335 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.714423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.714716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.715006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/1037a6248c8241828857d69fba1dce7d/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.717413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=1037a6248c8241828857d69fba1dce7d, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.718912 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.740542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 401/821] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:40 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/1037a6248c8241828857d69fba1dce7d/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:40.742083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.786730 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ca4d-bcfb-41e1-a479-348c7d97f606 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.839374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.840449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.840878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.841267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.843295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:40.843754 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.845802 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.874972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 421/822] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:40.875716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:40.904817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa4684-26e7-4fa4-9267-07b68cd218d1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:40.910547 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.950626 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:40.951554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:40.951849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:40.952134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/4daedcbeedb148b19078761a34b8d4a1/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:40.968794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:40.974379 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=4daedcbeedb148b19078761a34b8d4a1, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:40.975871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:40.993278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 402/823] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:40 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/4daedcbeedb148b19078761a34b8d4a1/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:40.994896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:41.027827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caf3a1-a4c1-4651-8e41-e70d9957dda3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.079869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.080526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.080823 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.081673 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.082910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] RBAC: Adding query filter params (name=network) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:41.083236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.085272 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.093200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 422/824] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:41.100804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:41.143155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58fa59-2f7c-40dd-b24e-8be4624266fc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.205518 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.206306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.206601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.206888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.216515 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:41.238228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251406b5-e898-407d-aedb-1ec5f0a6a181 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.279264 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.279937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.280226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.280540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.281277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] RBAC: Adding query filter params (type=network) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:41.281592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.283084 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.283670 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:41.294678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 423/825] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:41.296051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:41.301391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.303504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.412893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc8996-9199-46e0-974e-972c756a6e09 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.425906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 403/826] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:41.433976 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:41.485168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.485958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.486254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.486537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.487318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.488752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.535697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:41.536032 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 424/827] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:11:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:41.550457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067495a-bd17-4a88-b463-cee1800477dc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.599885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.600784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.601125 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.601410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.614514 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:41.637647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:41.639816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:41.641280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:41.641280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:41.641280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:41.641280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:41.641280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 404/828] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:41.647865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:41.670009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:41.813407 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133ff8b-e57f-451f-8c3c-50985ece5594 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:41.868534 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:41.869522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:41.869981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:41.870270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:41.871020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:41.871334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:41.872752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:41.895192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 425/829] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:41.901107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:42.075409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b643a-9fb5-41cb-a2ca-d4a923f09fd3 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:42.116106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:42.116972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:42.117269 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:42.117554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:42.121826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:42.126460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:42.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:42.144286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:42.145351 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:42.146311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 405/830] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:42.153914 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:42.243218 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:42.287503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b4e19-fa8f-400d-8d67-36807f48072a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:42.334787 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:42.335388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:42.335736 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:42.336018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:42.336928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:42.337245 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:42.339059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae12c57-5060-4bcc-890d-af7a86f2828a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:42.351650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 426/831] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:42.358724 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:42.476245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7381b1-8855-426d-85fb-d66100671f91 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:42.514187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:42.515294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:42.515791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:42.516348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:42.517458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:42.517981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:42.519662 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9de597-6508-469d-8ce8-4edafa0cf9aa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:42.527610 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 406/832] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:42 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:43.285109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed8e8b-8441-42ca-ac1c-90a9f2b4476f None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.285728 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed8e8b-8441-42ca-ac1c-90a9f2b4476f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.286020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed8e8b-8441-42ca-ac1c-90a9f2b4476f None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.287809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 427/833] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:43.294033 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33674f9-fc80-4fd6-90d4-ea2fd2befb28 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.294306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33674f9-fc80-4fd6-90d4-ea2fd2befb28 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.294607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33674f9-fc80-4fd6-90d4-ea2fd2befb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.313514 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-c33674f9-fc80-4fd6-90d4-ea2fd2befb28 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:43.314764 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-c33674f9-fc80-4fd6-90d4-ea2fd2befb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:43.376831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:43.392596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 407/834] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:43.414609 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed8e8b-8441-42ca-ac1c-90a9f2b4476f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:43.521925 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022f04-24f7-4297-9037-e07cd62e7b8b None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.522143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022f04-24f7-4297-9037-e07cd62e7b8b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.522401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022f04-24f7-4297-9037-e07cd62e7b8b None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.524224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 408/835] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:43.530244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98044-1ac4-49fe-91a1-01525f788d0e None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.530535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98044-1ac4-49fe-91a1-01525f788d0e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.530819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da98044-1ac4-49fe-91a1-01525f788d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.547711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed8e8b-8441-42ca-ac1c-90a9f2b4476f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:43.561420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-4da98044-1ac4-49fe-91a1-01525f788d0e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:43.562682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-4da98044-1ac4-49fe-91a1-01525f788d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:43.590935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:43.600292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:43.601612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.601909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.602190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.603196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:43.605211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:43.627411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:43.628650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:43.629552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 428/836] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:43.637463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:43.664435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 409/837] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:43.664856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7b48-b8d3-4c48-95cb-e9a5fffaf1e3 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.665211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7b48-b8d3-4c48-95cb-e9a5fffaf1e3 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.665559 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7b48-b8d3-4c48-95cb-e9a5fffaf1e3 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.671777 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 410/838] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:43.673170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.673507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.673800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.695947 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:43.697886 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:43.764574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4d35d-1b19-4877-9c15-42f392d3b88f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:43.780689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 411/839] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:43.781470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:43.811672 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:43.812351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.813129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.813414 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.814138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:43.814455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:43.815940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:43.826128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 429/840] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:43.826128 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:43.856170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:43.894686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:43.912439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b71df-034e-4e0e-a36c-68f156de6def None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:43.943281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c20ad4-c31d-4dfd-a4a8-70b1907af843 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:43.951208 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:43.951980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.952270 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.952583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:43.963913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:43.971885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:43.973018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:43.973940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 412/841] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:43.974690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:43.987831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:43.988994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:43.989604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:43.989887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.001301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:44.009230 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:44.011308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 430/842] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:11:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:44.012908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.098275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5326c1-7c99-444f-8bf3-ebc0ba51c0b9 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.109664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.129325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bafe4-4e89-4236-a8e0-e61ca9fbc600 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.139812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.140565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.140855 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.141141 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.141863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:44.142174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.143636 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.156423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 413/843] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:44.156423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.161690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.180588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.182146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.182671 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.182963 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.184009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:44.184347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.186037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.199458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 431/844] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:44.205691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.285549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e768398-dd9c-445a-befc-7fd2a39e8016 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.326603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.327332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.327623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.327904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.328720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:44.329036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.330461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.346012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 414/845] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:44.347467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.348197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af71c0d-4d7a-494a-95c4-dd08c53e8ecc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.388453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.402869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.404752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.404752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.404752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.417455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.418928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:44.442417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:44.443497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:44.444513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:44.444513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:44.444513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:44.444513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:44.447067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 432/846] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:44.453803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.481052 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581453e-62c9-4779-b539-dbb23ec7d25d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.498483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.520443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.521563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.521692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.521965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.534905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.536493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:44.557983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:44.559029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:44.559980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:44.559980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:44.559980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:44.559980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:44.564007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 415/847] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:44.565525 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.595552 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5220e-2b49-4731-b071-0b47d7420aff None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.635100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.635894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.636241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.636571 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.637314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:44.637634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.639079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.658466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 433/848] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:44.659792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.686510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.711961 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811ea9d-cc78-4230-8add-99bb58c63d05 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.758402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.759032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.759475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.760122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.761174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:44.761607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.763283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.782021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:44.782642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 416/849] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:44.784646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.788100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f10d69-0383-4319-9de9-7aedc7d49098 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.826136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.826943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.827228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.827504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] PATH_INFO: `/v3/users/swift` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.835496 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.836736 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:44.852864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:44.854442 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:44.855983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 434/850] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:44.861035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:44.917171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f287013-bc10-49cc-85d5-b18ae0dfa2ec None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.956248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:44.957189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.957833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.957833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:44.968371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:44.970211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:44.985108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614aa7c-84e7-468c-9cd7-f9c13b68ad4f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5de7b344-9767-47ee-8784-dbc04c48b388 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:44.993059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:44.994694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:44.995910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 417/851] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:44.997177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71c84-ef9c-46c2-ba74-3e5a7c02f2a1 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:44.997177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71c84-ef9c-46c2-ba74-3e5a7c02f2a1 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:44.997177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c71c84-ef9c-46c2-ba74-3e5a7c02f2a1 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.002753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 418/852] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:45.002753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c71c84-ef9c-46c2-ba74-3e5a7c02f2a1 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.011973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.044282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.045013 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.045301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.045579 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.046435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:45.046752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.048196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cbdb4-50d5-4d19-a41e-99ac529a4743 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.074406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 435/853] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:45.076008 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.076313 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.076717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.094106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:45.095543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:45.098600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.117576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c71c84-ef9c-46c2-ba74-3e5a7c02f2a1 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.161801 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.162495 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.162786 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.163070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.163934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:45.164257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.166337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.176112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 436/854] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:45.177851 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.201637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 419/855] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:11:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:45.201637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.274320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.302314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab9f08-98e4-40a8-93ef-4059aa2f4154 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.334554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed7048-66ce-485a-804f-3767dd5f3c15 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.345138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.346081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.346575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.347004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.359337 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.381423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.382262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.382551 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.382831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.383515 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.389257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:45.404871 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:45.406095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:45.407028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 420/856] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:45.407982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.444791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.446294 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.538401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28d472-7052-4445-90a9-a6e80ba22a3e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 437/857] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:45.569162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.569931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.580897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.581874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.582352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.582634 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.615409 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.662721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.664308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.686743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904ce72-5871-47c0-be9a-5d3e58cf94de None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.727523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.728737 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.729030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.729384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.730104 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:45.730416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.732038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.741882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 438/858] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:45.741882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.808449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 421/859] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:45.809208 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.841161 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.870305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce2f73-891c-4003-bd8f-2c4cb469c53c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.873947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.923670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.923670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.923670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.923670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.929929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252766f0-af26-4335-99eb-6f147004f5d1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:45.934147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.935723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:45.955741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:45.956992 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:45.958072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:45.958072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:45.958072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:45.958072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:45.959794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 439/860] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:45.960874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:45.975104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:45.976047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:45.976526 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:45.976967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:45.977923 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:45.978239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:45.980241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:45.996431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 422/861] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:45.996431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.124901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018afb76-91da-46db-9d84-05ee1d5ac0cd None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.132058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.143223 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebc0a-8c97-46bc-b17f-7cb9325c65af None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.181209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.182961 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.183323 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.183671 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.185781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.186770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.187251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.187732 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.188708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:46.189336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.190987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.209738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.211705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.216209 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 423/862] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:46.217314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:46.238124 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:46.239106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:46.240068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:46.240068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:46.240068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:46.240068 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:46.242049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 440/863] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:46.248287 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.261161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.355458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d30875-0641-4639-a7c0-aa95a37c26ff None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.398228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.399063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.399354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.399475 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5763a028-a1af-43a7-91d3-af27114e479a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.399838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.409772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.411792 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:46.432467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:46.433706 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:46.434680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:46.436588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 424/864] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:46.440070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.456523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.457244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.457542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.457821 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.458630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:46.458862 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.460249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.460966 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.479517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 441/865] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:46.481171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.554824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.594033 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ca19a-b940-427a-859f-7fdd8b1127f2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.606645 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e51d0-3851-458e-b613-6cc548a5d9c8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.629438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.630121 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.630404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.630708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.631523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:46.631840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.633388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39f996-2340-4f10-ad1b-d0353219eb50 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.651609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 425/866] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:46.651609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b8e-b24e-41c8-8e66-5faa6ac2e9ea None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.651609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b8e-b24e-41c8-8e66-5faa6ac2e9ea None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.651609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b8e-b24e-41c8-8e66-5faa6ac2e9ea None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.651609 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 426/867] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:46.652739 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2b8e-b24e-41c8-8e66-5faa6ac2e9ea None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.661065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.662662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.663187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.663700 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.674200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.676461 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ecf3d250-d796-4397-bf55-c4bc1d8ce8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:46.695019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:46.702256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 442/868] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.703217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.703884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.721449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:46.722659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:46.741473 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.777107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2b8e-b24e-41c8-8e66-5faa6ac2e9ea None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.802017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 443/869] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:46.803045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.832390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:46.833038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:46.833329 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:46.833612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:46.834469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=1037a6248c8241828857d69fba1dce7d) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:46.835003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:46.836607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:46.844361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 427/870] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/role_assignments?user.id=1037a6248c8241828857d69fba1dce7d&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:46.845232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:46.906891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:46.967508 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f4e29-e576-484f-8c1b-de003c200583 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:46.986348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e3b3d-ebd5-4b37-a38f-4e28856ac828 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:47.020182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:47.021880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:47.022659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:47.023142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:47.024486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:47.026043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:47.028550 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:47.034619 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:47.035445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:47.035814 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:47.036169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:47.037023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:47.039320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b348dd-da6c-400a-819a-80842a81c7b0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:47.043971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:47.048484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 444/871] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:47.054400 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:47.058621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 428/872] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:11:46 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:47.180689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856e959-3d38-43b3-9661-7556851554a9 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:47.204627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:47.234634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:47.235319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:47.235604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:47.235878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:47.236813 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=4daedcbeedb148b19078761a34b8d4a1) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:47.237132 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:47.238666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563427bb-2147-4139-a7bd-0afd7aff36c7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:47.247009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 445/873] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:47 2022] GET /identity/v3/role_assignments?user.id=4daedcbeedb148b19078761a34b8d4a1&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:48.386048 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32392455-3447-44f6-9a59-d577241c5c5a None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:48.386629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32392455-3447-44f6-9a59-d577241c5c5a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:48.386629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32392455-3447-44f6-9a59-d577241c5c5a None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:48.388384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 429/874] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:48.394340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c2d2-0308-4391-8e99-17894f998b61 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:48.394831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c2d2-0308-4391-8e99-17894f998b61 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:48.394914 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c2d2-0308-4391-8e99-17894f998b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:48.420371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-61a7c2d2-0308-4391-8e99-17894f998b61 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:48.421938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-61a7c2d2-0308-4391-8e99-17894f998b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:48.496238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 446/875] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:48.518934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32392455-3447-44f6-9a59-d577241c5c5a None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:48.633107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32392455-3447-44f6-9a59-d577241c5c5a None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:48.671774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:48.672854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:48.673442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:48.673957 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:48.675188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:48.677431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfa98c-50e4-4a86-bd90-d8a7ae7a9f57 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:48.694989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 430/876] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:11:48 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:49.324697 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b787b-2a48-4500-a7b9-6a590b146c38 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.325057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b787b-2a48-4500-a7b9-6a590b146c38 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.325269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b787b-2a48-4500-a7b9-6a590b146c38 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.327006 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 447/877] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:49.332704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2ea7-7354-4fcb-8a8f-0b076cddb4e9 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.333005 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2ea7-7354-4fcb-8a8f-0b076cddb4e9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.333296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb2ea7-7354-4fcb-8a8f-0b076cddb4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.352423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-39bb2ea7-7354-4fcb-8a8f-0b076cddb4e9 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:49.353678 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-39bb2ea7-7354-4fcb-8a8f-0b076cddb4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:49.410818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:49.437035 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 431/878] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:49.459750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b787b-2a48-4500-a7b9-6a590b146c38 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:49.501367 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb3a1-f995-4afc-a354-2e1a5128cf53 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.501837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb3a1-f995-4afc-a354-2e1a5128cf53 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.502222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cdb3a1-f995-4afc-a354-2e1a5128cf53 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.504113 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 432/879] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:49.510374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.510641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.511088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.529931 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:49.531341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:49.576002 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b787b-2a48-4500-a7b9-6a590b146c38 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:49.586829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:49.614045 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 433/880] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:49.620596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:49.620596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.620596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.620596 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.620954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:49.620954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:49.633766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:49.635670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.636812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 448/881] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:49.644704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:49.645535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:49.761625 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf7ffd-4848-4c07-b8ad-023ce2498e63 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:49.778784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:49.790297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadbfa8-1ed5-4878-9b71-6003a070ca47 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:49.804220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:49.805289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.805768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.807210 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.808187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:49.808758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:49.810381 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:49.820885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 434/882] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:49.823196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:49.832498 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:49.833903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.834024 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.834363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.835263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:49.836937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b7135645-6c78-42d2-a9ab-4d16d48c1b68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:49.850495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:49.851622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:49.852586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:49.854912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 449/883] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:49.856215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed4f5-aab1-4c31-93de-c2f5e9fbb366 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.856532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed4f5-aab1-4c31-93de-c2f5e9fbb366 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.856817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed4f5-aab1-4c31-93de-c2f5e9fbb366 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.859059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 450/884] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:49.860785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ed4f5-aab1-4c31-93de-c2f5e9fbb366 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:49.907527 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:49.948112 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13d32-d15d-4082-9e69-221b275f54d2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:49.988813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:49.989507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:49.989798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:49.990081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:49.990825 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:49.991138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:49.992349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ed4f5-aab1-4c31-93de-c2f5e9fbb366 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:49.992587 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded3cbb-d9ca-4118-a487-9aab5765de62 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:49.997498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 435/885] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:50.001850 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.002393 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.002831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.026215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:50.027792 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:50.046236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.047080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.047476 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.047867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.048703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:50.049139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.050915 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.072384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 451/886] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:50.072384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.083628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.131512 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 436/887] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:50.156605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.213365 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.218254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b253725-2d2e-4064-a7f4-61179299f683 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.277808 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.279382 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.280351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.281061 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.282722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:50.283501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.286269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.297417 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 452/888] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:50.302535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.321278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18e16f-0566-4eec-93bc-23ee711acb8f None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.360561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.361398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.361691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.361977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.375927 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:50.387643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:50.388986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:50.390398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 437/889] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:50.395028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.421636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdec1e-f304-46f4-8dbc-c2ecfff982ee None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.437334 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.472372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.473123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.473436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.474154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.475331 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] RBAC: Adding query filter params (interface=public, service_id=c798760608334a17853a55608730116b, region_id=RegionOne) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:50.475663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.477356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.489945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 453/890] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/endpoints?service_id=c798760608334a17853a55608730116b&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:50.495165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.530086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.537034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3d52f-f8e6-4228-9352-ca4de2221862 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.583708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.585382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.585883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.586382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.588428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:50.588998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.591290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.604294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 438/891] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:50.609506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.614958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660801d-b674-4a34-967e-175483b56c6a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.662750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.663938 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.664470 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.666196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.667295 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.669804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.678836 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 454/892] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:11:50.739720 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.750846 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278f21a-dc0b-44ea-8cf0-9b22e9585762 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:50.791092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:50.791823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:50.792105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:50.792380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:50.805161 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:50.807721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:50.827837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:50.828983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:50.830487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:50.830487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:50.830487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:50.830487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:50.830487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 439/893] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:50.835102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:50.951978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:50.962361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e2492-33cc-4516-9c47-c854bd60146e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.000603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.001286 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.001575 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.001851 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.002591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:51.002901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.004289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fa3af-fd21-4a9f-93b6-10b6690a8af4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.018153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 455/894] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:51.023770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.063212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0768-92ef-4987-a8da-2d40c05dde8c None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.063434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0768-92ef-4987-a8da-2d40c05dde8c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.063693 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0768-92ef-4987-a8da-2d40c05dde8c None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.065554 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 456/895] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:51.074204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.074509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.074801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.093935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:51.095270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:51.152915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7003c-bbbf-44f1-aa8b-34783972c9bc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.198827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 457/896] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:51.207863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.208721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.209008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.209288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.217847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.219428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.222902 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:51.242896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:51.244607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:51.246156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 440/897] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:51.253575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.260216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.339467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.366652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9348-5115-43c4-8a5a-4a284d2efdc3 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.402362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec3c06-d62f-4a78-8165-d2edbbef7eae None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.410792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.411612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.411904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.412189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.412907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.414292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:51.427037 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:51.428120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:51.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 458/898] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:51.437690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.448338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.449843 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.450395 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.450676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.451694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:51.452086 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.453701 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.474448 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 441/899] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:51.480333 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.548944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.557981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bed5-0170-4755-9fef-f8dbc38c9145 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.598336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339608-cbf7-40b6-a993-863ef40ddcf6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.600909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.602421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.602727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.603537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.604278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:51.604725 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.606135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.612448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 459/900] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:51.617844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.635626 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.649620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.650386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.650741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.651148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.738104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.740088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.762681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c38d39-2226-427b-91b4-2ddfc58c374c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:51.802750 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:51.803824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:51.804334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:51.804896 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:51.805832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:51.807162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:51.808836 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:51.815151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 460/901] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:51.820893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.825719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.872761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 442/902] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:51.877821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:51.927892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:51.968511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2062c-356b-471c-9b4d-1d13e26310eb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.007593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.007863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07822270-4067-4db2-9fcb-5d675ac495b8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.008878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.009929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.010427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.011490 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] RBAC: Adding query filter params (interface=public, service_id=9fa73bac1891459197d32bca69664d2a, region_id=RegionOne) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:52.012025 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.013647 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.019920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 461/903] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/endpoints?service_id=9fa73bac1891459197d32bca69664d2a&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:52.025085 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.057182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.058073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.058357 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.058633 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.073476 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.076537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:52.098475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:52.099513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:52.101132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:52.101132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:52.101132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:52.101132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:52.101132 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 443/904] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:52.109444 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.132869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:52.147735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91c9f9-d233-4f26-b920-64d5915327f4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.191865 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.192127 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.192565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.192565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.195467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.195467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.201643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 462/905] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:11:52 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:11:52.261359 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65be804-4336-4a94-96a6-bcbaccb14098 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.308477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.309182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.309474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.309759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.310528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:52.310844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.312286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b560d53-4e99-4eb4-a413-ed026ad10857 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.328200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 444/906] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:52.333964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.338074 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:52.457399 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda08b8-2d4c-4a04-ad11-e3de5ea907ba None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.494556 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.495356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.495638 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.495914 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.502980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.504444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.512209 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bd3e-e804-49f4-92e2-6baacbec03ad None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.512746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bd3e-e804-49f4-92e2-6baacbec03ad None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.512869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17bd3e-e804-49f4-92e2-6baacbec03ad None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.514454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 445/907] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:52.520458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:52.521569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:52.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:52.523420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.523420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.523420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.523629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 463/908] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:52.529561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.542657 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:52.543882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:52.596918 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:52.625859 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 446/909] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:52.650473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e941741-4fd6-40ec-acfb-ec442a2578f7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.653179 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.657476 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:52.699092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.700262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.700809 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.701347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.702398 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:52.703022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.704732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.724563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 464/910] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:52.726236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:52.784572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d48993-b5c6-4343-b4b2-2ed0b4ffc76c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.826063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.826759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.827050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.827338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.828458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.829750 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f75d8c-d3b5-4a22-8f49-72d02bf8ab04 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.847643 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4324ca8-c05e-4fda-90a4-113a89980b83 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:52.856897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 447/911] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:11:52 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:52.876695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:52.900933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:52.901616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:52.901905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:52.902188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:52.903039 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] RBAC: Adding query filter params (role.id=adfaa919206d419c9f04614ea0f4d9ff, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=4daedcbeedb148b19078761a34b8d4a1) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:52.903452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:52.904966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae4c90-c151-4c66-8164-2f5c7b933820 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:52.912129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 465/912] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:11:52 2022] GET /identity/v3/role_assignments?user.id=4daedcbeedb148b19078761a34b8d4a1&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=adfaa919206d419c9f04614ea0f4d9ff => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:53.216665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349547e-29d3-4121-9227-d10cb9f0212c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:53.217348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349547e-29d3-4121-9227-d10cb9f0212c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:53.217867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349547e-29d3-4121-9227-d10cb9f0212c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:53.219772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 448/913] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:53.225625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:53.225920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:53.226202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:53.244835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:53.246033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:53.323324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 466/914] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:53.346255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349547e-29d3-4121-9227-d10cb9f0212c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:53.419251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:53.464569 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349547e-29d3-4121-9227-d10cb9f0212c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:53.502834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:53.503647 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:53.503934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:53.504211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:53.504921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:53.506315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:53.519259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:53.521095 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:11:53.522023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:53.528432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 449/915] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:53.528715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:53.637484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ba131-924a-4292-b0eb-a13d352cc5e7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:53.652341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:53.675063 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:53.675810 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:53.676103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:53.676388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:53.677164 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:53.677483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:53.678907 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:53.685842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 467/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:53.691754 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:53.804560 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c23653-b56c-4b38-9f32-1f67a6a049d5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:53.837043 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:53.844043 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:53.845154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:53.845662 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:53.846153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:53.847111 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:53.847632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:53.849418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cbf67-a5db-43d0-8206-1204f2a1e452 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:53.860169 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 450/917] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:11:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:53.862005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:53.976487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602f1d8-3122-4915-bd58-30bac89c8697 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:54.014679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:54.015361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:54.015653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:54.015937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:54.016778 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:54.018216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4bbcc-b5a6-453b-8ade-6c3990a64f78 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:54.038791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 468/918] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:11:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:54.607806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ec8d-927a-44f1-b08f-5597bd407938 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:54.608809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ec8d-927a-44f1-b08f-5597bd407938 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:54.609307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3ec8d-927a-44f1-b08f-5597bd407938 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:54.611229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 451/919] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:54.616312 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:54.616811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:54.617267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:54.638199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:54.640123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:54.669020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:54.725357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 469/920] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:54.749690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ec8d-927a-44f1-b08f-5597bd407938 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:54.866317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3ec8d-927a-44f1-b08f-5597bd407938 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:54.906127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:54.906960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:54.907316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:54.907686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:54.908437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:54.909848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:54.923740 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:54.924866 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:54.925794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 452/921] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:11:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:54.933984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.048135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29dff9-7439-4c26-a9cd-056c85396215 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.087058 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:55.088382 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.088703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.088986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.089726 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:55.090036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:55.091450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:55.097738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 470/922] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:55.105527 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.167386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.220688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cb4ef-beec-4518-bbe1-c2d0d819cb8c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.264075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:55.264793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.265088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.265370 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.266109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:55.266425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:55.267821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e15b1-0926-4e67-93de-9e9d27ffb7f0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:55.273611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 453/923] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:55.279381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.308824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.399526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414feb8a-8a78-4ffd-934a-0564828ebfb2 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.407154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9443-a99f-492b-a214-d4bb82097e84 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.407529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9443-a99f-492b-a214-d4bb82097e84 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.407732 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9443-a99f-492b-a214-d4bb82097e84 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.409460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 454/924] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:55.415019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.415849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.416306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.435779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:55.436952 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:55.439096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:55.439780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.440062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.441073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.441199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:55.442510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363d092-5607-471c-8b75-41988af4c7c6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:55.476440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 471/925] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:11:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:55.476440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e04d62-bd50-4ba6-915c-066890952833 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.476440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e04d62-bd50-4ba6-915c-066890952833 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.476440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e04d62-bd50-4ba6-915c-066890952833 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.476440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 472/926] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:55.478234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.478531 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.478817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.497964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:55.499211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:55.548578 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 455/927] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:55.576571 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.604562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 473/928] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:55.627016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.638918 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.650317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.727263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b62b7-65cb-4ff5-a7c4-e62fa0976d08 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.761965 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46581efb-0216-4b8e-903f-57af00d856e3 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.770533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:55.771653 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.772156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.772467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.790658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:55.799489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:55.800616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:11:55.801536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:55.808505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 456/929] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:11:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:55.810027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:55.811162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:55.811460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:55.811744 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:55.812440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:55.812781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.813910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:55.833175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:55.834339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:55.835912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 474/930] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:55.841546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:55.950311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85522af-5c13-4b1e-ac35-80c12947b31f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:55.951003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.960509 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:55.993121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b6427-dd59-4ed6-8d4b-db9453ab19ef None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.000457 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.001174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.001465 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.001749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.003050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.003367 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.004813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.020462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 457/931] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:11:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:56.025695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.049065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.050768 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.051296 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.051756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.052853 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.053411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.056036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.069980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 475/932] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:56.070925 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.175474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb3837-d564-4d0f-b095-116daa849e7d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.204319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520b085-cc8a-4137-b44e-eead521ae091 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.230404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.231219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.231505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.231784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.247211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.248293 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.248822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.249455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.250411 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.250933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.252683 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.253660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.255374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.256172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.266216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 476/933] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:56.267707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:56.278097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:56.279099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:56.280065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:56.280065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:56.280065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:56.280065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:56.284119 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 458/934] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:56.285462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.313396 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.391769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a28ad-c391-43be-880b-64b18934b9b3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.413771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2349b89-607b-4c3f-a486-11d170f197c6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.432171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.433213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.433815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.434432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.435454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] RBAC: Adding query filter params (interface=public, service_id=a2548e2a0b974ad399b042e5be523317, region_id=RegionOne) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.435995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.437752 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.448836 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 477/935] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/endpoints?service_id=a2548e2a0b974ad399b042e5be523317&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:56.450224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.462967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.463998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.464292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.464612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.465554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.465811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.467504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.486239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 459/936] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:56.487629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.525278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.569109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b230c32e-3323-4e19-b36a-123d1bf1d31f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.588837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.605611 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.606301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.606591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.606872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.607573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.609013 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23836b66-2acd-4a95-a3e4-9657a8d86bf4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.611640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79171bd2-0893-4ee7-8de7-5340e9baf66e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.615612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 478/937] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:11:56.628444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf20d5-3a7c-41c0-950b-a25447bc74eb None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.628444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf20d5-3a7c-41c0-950b-a25447bc74eb None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.628444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf20d5-3a7c-41c0-950b-a25447bc74eb None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.628444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 479/938] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:56.630675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.630970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.631250 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.652201 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:56.653835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:56.661654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.663198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.663963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.664434 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.675479 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.677272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:56.699348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:56.700591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:56.701621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 460/939] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:56.707290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.742377 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 480/940] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:11:56.767341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.853824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb2eec-42ea-4fae-8a1a-16de3aa3ac52 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.858886 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.899235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.901466 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.901466 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.901466 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.901986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:56.902281 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.903885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.914377 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:56.916864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c126bf9-b4fd-449c-9df3-7fecf34919e5 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:56.933628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 461/941] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:56.939225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:56.965464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:56.966145 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:56.966436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:56.966723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:56.967777 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:56.969437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:56.970943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: WARNING keystone.server.flask.common [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:11:56.991805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.sql.core [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:11:56.991805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:11:56.991805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [parameters: {'id': '2180b78f33fb420380c48a597ee1a474', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:11:56.991805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214148) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2180b78f33fb420380c48a597ee1a474.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2180b78f33fb420380c48a597ee1a474. May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application conn.query(q) May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:11:57.001106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result.read() May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:11:57.002045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application next(self.gen) May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application yield resource May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application next(self.gen) May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise self.value May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application session.commit() May 19 20:11:57.003567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.session.flush() May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application compat.raise_( May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:11:57.004900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:11:57.005887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application conn.query(q) May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application result.read() May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:11:57.006769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [parameters: {'id': '2180b78f33fb420380c48a597ee1a474', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.008964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:57.011982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2180b78f33fb420380c48a597ee1a474. May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 481/942] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 20:11:56 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:11:57.016718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.078109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33ebf4-c352-4790-862e-3938c2e9ee03 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.118981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.120252 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.120895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.121440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.134914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.137155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.143651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40294d5d-6c7b-4123-8567-baf14ba6ebea None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:57.158006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:57.159094 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:57.160059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:57.160059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:57.160059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:57.160059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:57.162079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:57.169363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 462/943] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:57.172251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.186601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.187432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.187720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.188009 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.207197 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.209181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:57.226987 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:57.228090 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:57.229087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:57.237989 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 482/944] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:57.241162 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:57.242601 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.303286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da39dcfb-7938-416b-99df-49a770f23c9c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.345226 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.346500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.346500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.346929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.347796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:57.348224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.349800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.368905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 463/945] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:57.370472 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.372329 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bf9a6-fe74-436a-bbc5-b9f0c8a29a5f None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.413369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.414415 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.414926 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.415452 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.416696 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:57.417232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.419032 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.452732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 483/946] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:57.494315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731c5dc-837e-449e-9412-0b457f3d0bbc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.507301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:57.537736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.538687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.538981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.539266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.548903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.550474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:57.568531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:57.569564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:57.571157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:57.574963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 464/947] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:57.576484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.628019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:57.718814 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e1b7b-59a3-4350-9af1-bd6b548ab033 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.758637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.759377 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.759670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.760052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.760922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:57.761246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.762739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6982b0-d0f6-4c26-8618-92a4ce8b02bc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.777628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 484/948] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:11:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:57.782881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:57.911218 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a44ee-943e-4d55-90d4-d9c286b04189 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:57.957689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:57.959343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:57.960383 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:57.961160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/4daedcbeedb148b19078761a34b8d4a1/roles/adfaa919206d419c9f04614ea0f4d9ff` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:57.965917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=4daedcbeedb148b19078761a34b8d4a1, role_id=adfaa919206d419c9f04614ea0f4d9ff)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:57.969207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349ad19-d8a2-4419-9266-fa16ab2b385d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:57.987563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 465/949] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:11:57 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/4daedcbeedb148b19078761a34b8d4a1/roles/adfaa919206d419c9f04614ea0f4d9ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:58.145066 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa5797-8647-470f-ac28-f9f3d9a2f3df None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:58.145535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa5797-8647-470f-ac28-f9f3d9a2f3df None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:58.146048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa5797-8647-470f-ac28-f9f3d9a2f3df None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:58.149220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 485/950] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:58.158624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:58.159113 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:58.159608 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:58.196445 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:58.198284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:58.228975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:58.277102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 466/951] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:58.299653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa5797-8647-470f-ac28-f9f3d9a2f3df None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:58.406032 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:58.413852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa5797-8647-470f-ac28-f9f3d9a2f3df None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:58.452382 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:58.453127 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:58.453412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:58.453691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:58.454355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:58.455722 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:58.468773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:58.469888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:11:58.470798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 486/952] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:11:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:58.477862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:58.586111 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b4817-6cbd-4b56-9187-e9ddbcd38114 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:58.624727 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:58.625461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:58.625756 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:58.626041 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:58.626771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] RBAC: Adding query filter params (name=compute) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:58.627087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:58.628504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24215f2-386d-4439-8bad-54b034fc3b8f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:58.634075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 467/953] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:58.641669 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:58.755957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3c413-ed6b-4304-86cb-8ca4a822d2e0 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:58.796517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:58.797281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:58.797572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:58.797852 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:58.798699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] RBAC: Adding query filter params (type=compute) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:58.799105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:58.800581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4dc38-ddae-4209-88ae-41540835cc82 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:58.806422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 487/954] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:11:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:59.107906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d8f37-e702-4a85-84bc-bfc608dff518 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.108461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d8f37-e702-4a85-84bc-bfc608dff518 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.108652 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d8f37-e702-4a85-84bc-bfc608dff518 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.110439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 468/955] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:59.116279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.116618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.116900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.135146 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:59.136699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:59.154363 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:59.216510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 488/956] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:59.233628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d8f37-e702-4a85-84bc-bfc608dff518 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:59.260358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:59.355302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686d8f37-e702-4a85-84bc-bfc608dff518 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:59.396214 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:59.397072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.397360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.397642 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.398318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:59.399770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:59.412734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:59.413865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:11:59.414888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 469/957] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:11:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:59.420958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:59.534922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501e7fc-69c4-4300-adc2-acf9b60df6c3 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:59.571530 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:59.572273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.572595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.572881 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.573621 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:59.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:59.575365 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:59.581651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 489/958] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:59.587607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:59.685050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:59.700471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d9176-ef1b-4a54-a4ae-539a0f72d8aa None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:59.741534 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:59.742245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.745646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.745646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.745646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:59.745646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:59.746050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54adebd5-e515-49f7-9cd8-124a913bc117 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:59.752446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 470/959] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:11:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:59.758352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:59.785504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:59.870290 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83c266-cade-460e-b27c-2ea42e73b27c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:59.907405 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:59.908079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.908458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.908742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.909557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:59.910934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d7204-dc41-4bc6-9afc-f707105e6c9a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:59.933532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 490/960] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:11:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:59.960173 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded772b-3e2c-4c09-917e-088313205d4c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.960477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded772b-3e2c-4c09-917e-088313205d4c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.960721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded772b-3e2c-4c09-917e-088313205d4c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.963171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 471/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:59.969438 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:59.970080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:59.970548 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:59.990032 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:59.991832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:00.075392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:00.088788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 491/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:11:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:00.109529 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded772b-3e2c-4c09-917e-088313205d4c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:00.235968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded772b-3e2c-4c09-917e-088313205d4c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:00.276593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:00.277277 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.277561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.277840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.278551 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:00.280007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:00.281363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: WARNING keystone.server.flask.common [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:12:00.296811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.sql.core [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:12:00.296811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:12:00.296811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [parameters: {'user_id': 'bb43bb826f8243529cdf95789a561075', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:12:00.296811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=214149) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application conn.query(q) May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application result.read() May 19 20:12:00.315265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:12:00.316506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application next(self.gen) May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application yield resource May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application next(self.gen) May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise self.value May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application session.commit() May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:12:00.317607 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self.session.flush() May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application compat.raise_( May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:12:00.318552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:12:00.319510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application conn.query(q) May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application result.read() May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:12:00.321127 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bb43bb826f8243529cdf95789a561075', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:00.323301 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:12:00.325296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:00.327532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 472/963] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:12:00 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:00.328477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:00.438466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c6ddb-9dfa-4c4f-a788-fc93712252b5 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:00.477370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:00.478257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.478547 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.478820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.486986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:00.488387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-c0a323ff-50f3-4c9c-9c07-9676d8d9e6e9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:12:00.504305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:00.506606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:00.508919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 492/964] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:00.516451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:00.530035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93614e8-7850-46d6-bf2a-a49affed190e None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.530330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93614e8-7850-46d6-bf2a-a49affed190e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.530617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93614e8-7850-46d6-bf2a-a49affed190e None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.532349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 493/965] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:00.537099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:00.537318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.537523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.537806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.556299 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:00.557827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:00.624151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:00.641319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 494/966] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:00.664780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68bd335-3652-4cc8-a41f-4d2dd235618c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:00.667663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:00.739129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:00.740262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.740972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.741646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.743848 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:00.744471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:00.747847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:00.784142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60a93c-cf05-430e-aecd-3cefbe457653 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:00.799710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 473/967] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:12:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:00.835576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:00.836503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:00.836800 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:00.837148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:00.849877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:00.859257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:00.861603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:00.863614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 495/968] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:12:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:00.868976 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:00.960022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.002336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dce2f-e6c5-4633-aaac-ce320167608a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.056834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.058067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.058512 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.058947 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.059922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:01.060389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.061981 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e2524e-cc27-4be0-b32d-bcc0045b4374 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.076783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 474/969] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:12:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:01.078877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.143395 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.205239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b455b-1dbc-4e1f-b82c-0d6cce0aac2d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.242322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.243148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.243497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.243784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.256054 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.257478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.273874 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3ee6-2697-458a-a123-415bc566e2ae None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.274400 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3ee6-2697-458a-a123-415bc566e2ae None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.274486 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec3ee6-2697-458a-a123-415bc566e2ae None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:01.276130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:01.277252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:01.278220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:01.278220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:01.278220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:01.278220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:01.278462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 475/970] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:01.278539 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 496/971] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:01.282222 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.282535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.282787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.284829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.306794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:01.310496 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:01.402351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.403063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 476/972] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:01.407642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09fbd-226a-4645-98ca-5a7d5f4878ac None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.426284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.445351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.446082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.446358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.447520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.448210 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:01.448566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.450011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.471078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 497/973] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:01.472833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.503126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.545115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f93f6-2620-4737-ae41-0c2a7186d6e7 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.580081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68af2f-bc2b-4ee5-9f8b-3bc93fb71a91 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.582532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.583212 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.583499 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.583780 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.584539 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.585922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.600516 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 477/974] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:12:01 2022] POST /identity/v3/services => generated 242 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:12:01.636955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.637908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.638194 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.638522 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.647359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.648952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:12:01.667212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:01.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:01.675292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 498/975] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:12:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:01.678007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.796983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e1e59-14ec-4488-b357-d2235c435de4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:01.825428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.841983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:01.842682 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:01.842967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:01.843369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:01.844109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:01.844447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:01.845904 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:01.871047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 478/976] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:12:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:01.875684 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:01.896834 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:01.997739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949456b1-f5bb-4b02-ae55-1e9949d85c9e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.035565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.036232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.036569 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.036846 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.116391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.117831 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967b798-fe56-4a38-adf6-7e86def039ee None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.147267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:02.262150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 499/977] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:01 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:02.264659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.385459 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b120a27-c8f0-4af9-b8c8-278000f554fe None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.424952 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.425758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.426050 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.426337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.437845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.439321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.451226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07536-5f2d-442c-a699-093ed149403a None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.451473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07536-5f2d-442c-a699-093ed149403a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.451719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07536-5f2d-442c-a699-093ed149403a None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.453330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 500/978] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:02.458616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.458933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.459199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:02.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:02.468319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 479/979] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:02.474375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.481832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:02.483142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:02.519473 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:02.565937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 501/980] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:02.588957 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.607038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292c59-40d8-4117-b455-b995db6db9ef None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.640208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:02.655061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.655762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.656083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.656333 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.657136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:02.657453 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.658893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.674099 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 480/981] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:02.679674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.710073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c52121-1665-4625-8ccc-faf0493d4620 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.751904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.753169 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.753724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.754199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.755057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.756681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:02.774320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:02.776251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:02.777437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:02.780087 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 502/982] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:02.785926 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.799386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:02.808382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d01be-85c9-4002-a187-e198792dfdcb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.856618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.857426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.857713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.857988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.865762 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.867140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:12:02.882749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:02.883822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:02.885492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:02.889677 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 481/983] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:02.890659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:02.900929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee7f3d-4220-446e-89ef-1162cf134a8b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:02.913829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:02.943376 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:02.944086 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:02.944375 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:02.944803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:02.945640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:02.945956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:02.948024 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:02.953568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 503/984] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:02.959044 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:03.003320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f965d13-89a9-412d-8ea2-1abe6353893e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:03.042523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:03.043361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:03.043649 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:03.043934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:03.045337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:03.045337 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:03.048729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:03.060524 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 482/985] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:03.066336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:03.074063 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491fab7-e063-4886-88db-019a24c96666 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:03.083670 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:03.111537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:03.112213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:03.112540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:03.112824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:03.113546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:03.113863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:03.115241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:03.127620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 504/986] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:12:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:03.128655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:03.175368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:03.178223 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a25eb-8ec4-4206-a625-9f750f06575f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:03.221844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:03.222666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:03.223056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:03.223440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:03.224387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] RBAC: Adding query filter params (role.id=adfaa919206d419c9f04614ea0f4d9ff, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=4daedcbeedb148b19078761a34b8d4a1) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:03.224832 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:03.226800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:03.235183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 483/987] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:03 2022] GET /identity/v3/role_assignments?user.id=4daedcbeedb148b19078761a34b8d4a1&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=adfaa919206d419c9f04614ea0f4d9ff => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:03.251615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23261-4f68-4568-867f-2c475093d587 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:03.293354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:03.294196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:03.296911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:03.296911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:03.296911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] RBAC: Adding query filter params (interface=admin, service_id=a2548e2a0b974ad399b042e5be523317, region_id=RegionOne) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:03.296911 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:03.297904 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d088f0a-b3fb-40cd-ad53-07c23a437043 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:03.304541 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 505/988] 10.222.0.13 () {56 vars in 1249 bytes} [Thu May 19 20:12:03 2022] GET /identity/v3/endpoints?service_id=a2548e2a0b974ad399b042e5be523317&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:03.310131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:03.404042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:03.430618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323bc18-3fdd-4d6e-9035-a0091f610596 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:03.481266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:03.482205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:03.482680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:03.482984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:03.483813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:03.485174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b423-742b-4367-9f0b-b9e70934413b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:03.492159 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 484/989] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:12:03 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:12:04.013880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec3266-d100-4dd8-ac96-bd9c417a9ddd None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.014464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec3266-d100-4dd8-ac96-bd9c417a9ddd None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.014464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec3266-d100-4dd8-ac96-bd9c417a9ddd None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.016607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 506/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:04.022185 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.022494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.022757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.040625 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:04.042421 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:04.129708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 485/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:04.153294 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec3266-d100-4dd8-ac96-bd9c417a9ddd None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:04.277505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec3266-d100-4dd8-ac96-bd9c417a9ddd None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:04.279242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:04.316042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:04.316888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.317185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.317465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] PATH_INFO: `/v3/services/compute` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.318122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:04.320129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:04.332581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:04.333776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:04.334705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:04.336441 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 507/992] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:04.340725 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:04.414807 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:04.450057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc074-ab5c-4b42-a248-925e8bc974cf None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:04.488695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:04.489398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.489688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.489989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.490714 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:04.491036 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:04.492461 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:04.498084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 486/993] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:12:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:04.503539 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:04.610379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a25f50-e64a-40dc-b8e6-475bed052f20 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:04.645607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:04.646303 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.646587 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.646866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.647604 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] RBAC: Adding query filter params (type=compute) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:04.647916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:04.649361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:04.656107 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 508/994] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:12:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:04.661976 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:04.769808 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac19bc-b1b2-424f-834f-ce2e38e7d7bc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:04.806734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:04.807688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.807977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.808258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.809109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] RBAC: Adding query filter params (interface=public, service_id=f0bdd0693957475298d71d70272e10d0, region_id=RegionOne) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:04.809431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:04.810829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da823a-e9fa-4587-81a8-69825c26decc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:04.813617 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:04.822364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 487/995] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:12:04 2022] GET /identity/v3/endpoints?service_id=f0bdd0693957475298d71d70272e10d0&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:04.823603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:04.857120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:04.934222 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a4843-c776-414e-9672-9c2c3f27297a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:04.971131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:04.971806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:04.972092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:04.972369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:04.973096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:04.974464 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba4c71-f2c5-43fe-94d9-329b025a4856 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:04.988672 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 509/996] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:12:04 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:12:05.680106 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da089d-420e-4e40-b4a4-9a861f6b9c97 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:05.681294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da089d-420e-4e40-b4a4-9a861f6b9c97 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:05.681294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da089d-420e-4e40-b4a4-9a861f6b9c97 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:05.683489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 488/997] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:05.690005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2647e-2399-4b46-8810-18e18285b4af None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:05.690595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2647e-2399-4b46-8810-18e18285b4af None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:05.690595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2647e-2399-4b46-8810-18e18285b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:05.709639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-62a2647e-2399-4b46-8810-18e18285b4af None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:05.710847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-62a2647e-2399-4b46-8810-18e18285b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:05.792347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 510/998] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:05.815248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da089d-420e-4e40-b4a4-9a861f6b9c97 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:05.817052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1f79-1058-4db7-b8c1-cc4504aa4ed7 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:05.817350 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1f79-1058-4db7-b8c1-cc4504aa4ed7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:05.817633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c1f79-1058-4db7-b8c1-cc4504aa4ed7 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:05.819225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 511/999] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:05.824932 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:05.825223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:05.825515 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:05.842481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:05.843745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:05.863073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:05.924184 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 512/1000] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:05.935631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da089d-420e-4e40-b4a4-9a861f6b9c97 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:05.947348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:05.984315 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:05.985138 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:05.985426 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:05.985709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] PATH_INFO: `/v3/services/image` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:05.986399 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:05.987781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:06.000292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:06.001779 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:06.002768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 489/1001] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:12:05 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:06.007604 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:06.063211 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e89063-54d6-4580-af81-6e50d81fbcc2 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.079889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:06.109864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.110702 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.110986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.111266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.112080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.113623 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.120998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ed417-b3c2-40e5-8d98-09ff4af3146f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:06.126978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:06.128123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:06.129153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 513/1002] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:12:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:06.135820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:06.158339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:06.169764 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.170775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.171354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.171640 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.172563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] RBAC: Adding query filter params (name=image) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:06.172882 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.174827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.180065 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 490/1003] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:06.186855 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:06.268309 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c897b-0f78-4d1f-b95d-ecbc80d3342d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.307458 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.307728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eebd77-5fc9-4819-b112-5fc65e5eabbb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.308162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.308481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.308761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.309606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:06.309916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.311493 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.317654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 514/1004] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:12:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:06.324221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:06.336493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:06.347885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.348629 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.348899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.349178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.350046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:06.350264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.351613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.364454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 491/1005] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:06.425919 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:06.450595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e94c-9d3b-47c7-a526-d38cf34bc7c5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.490710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.491406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.491688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.491966 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.492724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:06.493038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.494629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc17638f-9da7-45e8-a6f8-32adc0ba86eb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.504942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 515/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:12:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:06.506272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:06.618013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5da55-ea60-4ea9-90c2-a5ffed9d14d0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:06.655402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:06.656078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:06.656362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:06.656672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:06.657450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:06.658838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d1da9-aa2f-4998-ad91-ae4cc6ab5b9d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:06.679071 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 492/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:12:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:12:07.373043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de91b-98c0-40f2-b3e4-dc548fc5ed1d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:07.373599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de91b-98c0-40f2-b3e4-dc548fc5ed1d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:07.373599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de91b-98c0-40f2-b3e4-dc548fc5ed1d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:07.375007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 516/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:07.380263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:07.380712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:07.380891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:07.397818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:07.399130 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:07.451455 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:07.484738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 493/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:07.508447 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de91b-98c0-40f2-b3e4-dc548fc5ed1d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:07.624653 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de91b-98c0-40f2-b3e4-dc548fc5ed1d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:07.664629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:07.665507 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:07.665798 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:07.666080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] PATH_INFO: `/v3/services/compute` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:07.666762 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:07.668207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:07.681716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:07.683177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:12:07.684151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:07.685955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 517/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:07.689352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:07.800531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f138b14-28d2-4334-b907-77e9c6d1338e None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:07.839075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:07.839758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:07.840048 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:07.840331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:07.841089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] RBAC: Adding query filter params (name=compute) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:07.841409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:07.842811 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:07.852954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 494/1011] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:12:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:07.854684 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:07.877997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:07.975662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ba6-5b7a-49fb-9b7a-c8220c0d7760 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:08.013174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:08.013845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:08.014185 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:08.014469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:08.015179 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] RBAC: Adding query filter params (type=compute) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:08.015489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:08.016887 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6125ca4-ea73-4966-bf85-1c98759182bb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:08.027310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 518/1012] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:12:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:08.028374 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:08.075320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:08.136450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cbe5c-e26d-4438-80cf-b1efd63e6539 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:08.173388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:08.174409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:08.174715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:08.175001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:08.175768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:08.177200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79343-d85a-4fa3-9ad6-8162f82f038a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:08.212674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 495/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:12:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:12:08.655282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69f61-ed30-4e5d-96d0-87f62b4fdbbf None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:08.656460 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69f61-ed30-4e5d-96d0-87f62b4fdbbf None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:08.656923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a69f61-ed30-4e5d-96d0-87f62b4fdbbf None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:08.658666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 519/1014] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:08.664787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052332f2-93fe-4bc0-ba87-15e796929504 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:08.665339 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052332f2-93fe-4bc0-ba87-15e796929504 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:08.665444 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052332f2-93fe-4bc0-ba87-15e796929504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:08.684594 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-052332f2-93fe-4bc0-ba87-15e796929504 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:08.685809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-052332f2-93fe-4bc0-ba87-15e796929504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:08.775482 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 496/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:08 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:08.797435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a69f61-ed30-4e5d-96d0-87f62b4fdbbf None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:08.885482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:08.914328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a69f61-ed30-4e5d-96d0-87f62b4fdbbf None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:08.950688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251e335-34f1-429d-85b3-731807373aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:08.951368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e335-34f1-429d-85b3-731807373aed None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:08.951652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e335-34f1-429d-85b3-731807373aed None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:08.951933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e335-34f1-429d-85b3-731807373aed None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:08.952707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251e335-34f1-429d-85b3-731807373aed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:08.954082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251e335-34f1-429d-85b3-731807373aed None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:08.976427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 520/1016] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:12:08 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:09.067462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3a31-ede0-4521-93b5-d718b645ed02 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:09.067803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3a31-ede0-4521-93b5-d718b645ed02 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:09.068038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c3a31-ede0-4521-93b5-d718b645ed02 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:09.069703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 497/1017] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:09.075903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:09.076244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:09.076521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:09.095067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:09.096301 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:09.182955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 521/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:09.209910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c3a31-ede0-4521-93b5-d718b645ed02 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:09.250460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:09.369628 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c3a31-ede0-4521-93b5-d718b645ed02 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:09.411505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:09.412372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:09.412684 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:09.412966 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:09.417112 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:09.418589 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf11a6-eb71-41e2-a259-e212c5bd1f4e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:09.426844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 498/1019] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:09.428046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:09.504932 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:09.565250 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf6d4b-afa8-4c87-97a3-b3d3ace4766b None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:09.603813 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:09.604553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:09.604844 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:09.605122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:09.605874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:09.607478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127f95b1-6df3-4d8d-bc34-27dbf7601ec7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:09.644099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 522/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:12:09 2022] POST /identity/v3/projects => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:10.770161 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2f14-8f37-41ba-98e8-7c8c76972361 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:10.770735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2f14-8f37-41ba-98e8-7c8c76972361 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:10.770735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d2f14-8f37-41ba-98e8-7c8c76972361 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:10.772244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 499/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:10.778956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:10.779436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:10.779538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:10.798519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:10.799739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:10.889095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 523/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:10.913971 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2f14-8f37-41ba-98e8-7c8c76972361 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.017932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.030245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d2f14-8f37-41ba-98e8-7c8c76972361 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.076676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.077505 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.077793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.078074 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.089279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:11.097543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:11.098668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:11.099586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:11.104494 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 500/1023] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:12:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:11.106158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.152838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.235618 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb943c-cf73-4773-9d5a-da1a53b57c9e None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.272565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.273243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.273527 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.273803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.275398 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:11.275677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.277108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dbf-8403-4935-b2d5-64088db42aee None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.290829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 524/1024] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:11.291996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.366207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9e6c-2ba5-4fcf-bb8a-bccbb7866efd None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.366563 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9e6c-2ba5-4fcf-bb8a-bccbb7866efd None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.366849 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9e6c-2ba5-4fcf-bb8a-bccbb7866efd None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.376879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 525/1025] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:11 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:11.383597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.384733 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.385067 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.408879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946afd89-cc2b-49dc-9dea-616756266ee8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.411094 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:11.412297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:11.445823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.447159 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.447510 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.447719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.459806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.461844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:11.487523 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:11.488816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:11.489969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:11.489969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:11.489969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:11.489969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:11.491681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 501/1026] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:11.497796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.500318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 526/1027] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:11.502735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.522502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.559589 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.614096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f035fe-7b6b-462f-bd31-a4fecf415df2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.637456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f042e-a5b1-404c-ac2d-b5224cb08d23 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.650715 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.651398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.651689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.651974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.652751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:11.653066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.654469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.672438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 502/1028] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:11.674205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.700620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.701418 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.701703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.701982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] PATH_INFO: `/v3/services/image` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.702649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.704033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:11.722385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:11.723544 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:11.724487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:11.729371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 527/1029] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:11.738758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.757675 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.814922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb293c-3b4c-44cb-9d99-2137789b7f35 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.857443 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.858247 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.858541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.858822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.866692 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:11.867171 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.868964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.871742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9e756-0455-4297-8686-816f63f749bf None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:12:11.887986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:11.889182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:11.891038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 503/1030] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:11.896117 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:11.912570 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:11.913244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:11.913532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:11.913811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:11.914523 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] RBAC: Adding query filter params (name=image) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:11.914839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:11.916489 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:11.926436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 528/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:11.927562 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.012761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc67-5501-408c-8352-6f2886278973 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.038508 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.051485 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.052183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.052501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.052793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.053664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:12.053977 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.055441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fbea1-ba15-4df6-9342-71819bb44c3c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.058880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e061-0dee-4110-9682-14c49523fc37 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.075481 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 504/1032] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:12.077080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530d73-e867-420c-b250-24a2ff9f612b None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.077375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530d73-e867-420c-b250-24a2ff9f612b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.077661 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530d73-e867-420c-b250-24a2ff9f612b None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.078916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 505/1033] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:12.081029 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77530d73-e867-420c-b250-24a2ff9f612b None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.100874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.101551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.101833 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.102106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.103113 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] RBAC: Adding query filter params (type=image) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:12.103448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.104855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1511d-a66a-496f-9f4b-623626d0e66f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.110000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 529/1034] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:12.110000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.110000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.110000 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.128016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:12.129238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:12.150958 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.188246 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77530d73-e867-420c-b250-24a2ff9f612b None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.200712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 530/1035] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:12.201614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.237227 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.237891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.238175 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.238456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.289632 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.319084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.319480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9426270-e481-4749-8e76-fcfb8e7ecf4b None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.320591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.356847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.357457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.357747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.358027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.358850 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] RBAC: Adding query filter params (interface=public, service_id=cb099d3ef420403fa4ff55414147c08f, region_id=RegionOne) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:12.359178 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.360615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.366649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 531/1036] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/endpoints?service_id=cb099d3ef420403fa4ff55414147c08f&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:12.367448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.390682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.430624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 506/1037] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:12:12.431920 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.485956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03f0e-8ca5-4c72-a652-14656ef3a2ed None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.522341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.523154 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.523439 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.523720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.525030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.526487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.528141 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 532/1038] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:12.528943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.534379 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.553847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85b9f-e485-461c-ae7c-0da29bd51a39 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.598860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.599531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.599815 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.600097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.600832 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.602220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.609935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 507/1039] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:12:12.610653 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.626675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.641657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf68d6-406a-43f9-b82c-717d8972db3d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.687259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.688093 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.688511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.688689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.701882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.703503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:12.724199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:12.725385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:12.726457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:12.726457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:12.726457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:12.726457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:12.726457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 533/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:12.731910 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.745402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985e553-90fd-4743-9de1-f5c5eaf29230 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.788416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.789146 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.789438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.789732 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.791089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.792771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.818969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 508/1041] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:12:12 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:12.867065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0a56-254e-4b71-b322-c5e021e8efd3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:12.910069 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:12.910764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:12.911055 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:12.911338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:12.912093 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:12.912428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:12.913912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:12.925033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:12.935748 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 534/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:12.937041 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:12.940014 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:13.059826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf931883-9b64-47c7-925b-53239ece1c3d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:13.098665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:13.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:13.099782 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:13.100056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:13.108268 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:13.109790 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:12:13.126416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:13.127540 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:13.128967 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 509/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:13.133330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:13.253017 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1879d4-f54d-4ad5-8806-1783aac4dfbd None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:13.289478 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:13.290797 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:13.291147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:13.291431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:13.292298 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:13.292663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:13.294136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8041242-393c-4524-a46f-3262b9d59f37 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:13.308269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 535/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:13.314066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:13.380622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:13.423646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66953d48-ab77-4b6a-98a0-db97778b74da None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:13.463003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:13.463708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:13.463949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:13.464224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:13.465120 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] RBAC: Adding query filter params (role.id=adfaa919206d419c9f04614ea0f4d9ff, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:13.465434 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:13.466887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470264ee-65cd-4d16-a328-d4bc1fb65de6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:13.480694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 510/1045] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:13 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=adfaa919206d419c9f04614ea0f4d9ff => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:14.992848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a7880-aeb3-4584-aecc-ca52c26f1a80 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:14.993614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a7880-aeb3-4584-aecc-ca52c26f1a80 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:14.993614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a7880-aeb3-4584-aecc-ca52c26f1a80 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:14.994879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 536/1046] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:15.001829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c307ec-06bd-48af-aee5-54d778c3f690 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.002296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c307ec-06bd-48af-aee5-54d778c3f690 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.002650 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c307ec-06bd-48af-aee5-54d778c3f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.026879 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-13c307ec-06bd-48af-aee5-54d778c3f690 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:15.029053 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-13c307ec-06bd-48af-aee5-54d778c3f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:15.160584 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 511/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:15.186026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a7880-aeb3-4584-aecc-ca52c26f1a80 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.270040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:15.286995 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296138-7657-4493-8cab-bac1d98338c7 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.287613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296138-7657-4493-8cab-bac1d98338c7 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.287992 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f296138-7657-4493-8cab-bac1d98338c7 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.290122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 512/1048] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:15.295993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.296296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.296676 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.317042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:15.318341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:15.327423 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a7880-aeb3-4584-aecc-ca52c26f1a80 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.371297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.372057 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.372341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.372646 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] PATH_INFO: `/v3/services/image` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.373306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:15.374716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:15.382997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:15.391982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:15.393225 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:12:15.394165 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:15.396283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 537/1049] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/services/image => generated 86 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:15.401447 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.413150 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 513/1050] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:15.433861 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.545959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322aeedf-1414-4f0c-aacf-9d30935ab579 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.573782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:15.585185 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a04ce-cdcc-426f-8b8a-3d0465faf88d None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.587280 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.588310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.588699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.589056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.589794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] RBAC: Adding query filter params (name=image) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:15.590113 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:15.600091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:15.606627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 538/1051] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:15.612495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.625261 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.626401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.626993 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.627289 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.642125 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:15.658243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:15.659430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:15.660342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:15.661100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 514/1052] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:15.666887 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.728242 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:15.740358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84226d5d-47cd-442f-b896-e56dfbae5703 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.780834 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.781513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.781800 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.782082 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.782800 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] RBAC: Adding query filter params (type=image) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:15.783116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:15.784551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:15.789565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 539/1053] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:15.795745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.812429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e77016-8fd8-4bc8-ad34-7efd0c3e9117 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.851320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.852053 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.852359 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.852838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.854303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:15.854730 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:15.856230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:15.859560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:15.870564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 515/1054] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:15.871659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:15.917048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7e5a-bf22-429f-b3ca-04da8eaacdc5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.952681 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:15.953351 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.953642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.953922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] PATH_INFO: `/v3/endpoints` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.954735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:15.956216 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5830b9-f343-43ff-9b6e-ff8759a6aadc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:15.978424 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c3a10-8149-4aeb-8520-66e82ae3e8a6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:15.978928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 540/1055] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:12:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:15.980261 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964437e4-efc1-445a-a54c-2b94b2b067ec None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.980591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964437e4-efc1-445a-a54c-2b94b2b067ec None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.980882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964437e4-efc1-445a-a54c-2b94b2b067ec None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:15.982157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 541/1056] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:15.987917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:15.988207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:15.988517 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.004595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.011176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:16.013038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:16.052770 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.053947 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.054419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.054885 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] PATH_INFO: `/v3/users/ead082bbed5546e49dd9451966c0ccb8` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.075250 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] RBAC: Authorizing `identity:get_user(user_id=ead082bbed5546e49dd9451966c0ccb8)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.077386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.084735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 516/1057] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:15 2022] GET /identity/v3/users/ead082bbed5546e49dd9451966c0ccb8 => generated 285 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:16.086542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.097468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 542/1058] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:16.121529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.172035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.215189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfd295-e047-464d-a74a-a45e480d680f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.249072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb01a2-4b0e-4785-a6fa-6f77a0e8e8f7 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.255408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.256075 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.256362 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.257116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.289491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.290942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.291224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.291586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.293860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.303514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:16.312964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:16.314220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:16.315344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 543/1059] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:16.325180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.345814 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.347207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.441706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.445743 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af819b-4cf3-4d5e-9392-ffbb514c3678 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.465343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 517/1060] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:12:16.466502 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.490908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.491688 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.492040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.492389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.494016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:16.494448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.495920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.505484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 544/1061] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:16.512147 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.538495 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.583546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662449e-503c-4a16-857c-e1bb82c8672d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.622272 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.623500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.623781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.624055 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.625302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.626763 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.628507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 518/1062] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:16.634852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.640347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7d692-8ec8-4f66-8b87-719fba8f4d2d None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.694567 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.695390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.695687 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.695973 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.707715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.709247 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:16.729317 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:16.730557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 545/1063] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:16.736685 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:16.753984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.778803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ced85-7e78-41b0-9f8a-cc7e5b356752 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.814903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:16.821721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.822696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.823235 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.823648 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.824719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a68bcd8a62b440169c14271779b85430, user.id=ead082bbed5546e49dd9451966c0ccb8) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:16.825450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.827851 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.833959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 519/1064] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/role_assignments?user.id=ead082bbed5546e49dd9451966c0ccb8&scope.project.id=a68bcd8a62b440169c14271779b85430&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:16.873710 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29693485-62da-4a7a-81d2-859de5f9e14c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:16.919820 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:16.920566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:16.920861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:16.921144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:16.921913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:16.922229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:16.923650 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:16.951254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 546/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:16.954087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:17.081590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e70f5-413c-49f1-931c-70aa9f48a3fd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:17.116901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:17.117702 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:17.118646 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:17.118922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:17.131613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:17.133044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:12:17.148200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:17.149968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:17.165736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 520/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:12:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:17.167651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:17.248935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:17.291422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac7a0b-8f94-48af-9cbe-1bbc2116fd13 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:17.333557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:17.334251 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:17.334542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:17.334825 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:17.335683 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:17.336002 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:17.338091 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:17.361776 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 547/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:12:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:17.362795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:17.429775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:17.483097 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe966b5-58b3-4d33-8b76-16160543e5cc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:17.517577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:17.518389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:17.518831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:17.518948 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:17.531177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:17.532681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:17.550688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:17.557403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:17.558408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:17.558408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:17.558408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:17.558408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:17.558408 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 521/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:17.559100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:17.684619 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4ad90-dbc0-4096-bbd8-f689c8a6888c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:17.716780 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:17.724081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:17.724719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:17.725015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:17.725330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:17.726051 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:17.726354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:17.728026 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:17.744479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 548/1069] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:17.747160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:17.856231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31075d6c-7467-497d-84de-91f2405f695b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:17.863444 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:17.902734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:17.903549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:17.903847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:17.904131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:17.911139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:17.912644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:12:17.927276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:17.928278 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:17.935860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 522/1070] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:17.936483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:18.054330 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c851f0-3785-4ec7-89dd-7f1c5a0db370 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:18.090932 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:18.091644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.091937 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.092220 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:18.093108 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:18.093425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:18.094923 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fdbfa-b8ea-4a0d-836a-c27ac73025e2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:18.113319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 549/1071] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:12:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:18.114716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:18.248023 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f0795-53b5-4d1b-ba48-7d7251cca144 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:18.286265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:18.287163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.287450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.287731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/adfaa919206d419c9f04614ea0f4d9ff` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:18.290062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=bdccfb4336b7491d8797c3f682ea0687, role_id=adfaa919206d419c9f04614ea0f4d9ff)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:18.291564 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae31e15-5fae-4b59-a925-91ba81d8841a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:18.308929 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 523/1072] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:12:18 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/bdccfb4336b7491d8797c3f682ea0687/roles/adfaa919206d419c9f04614ea0f4d9ff => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:12:18.338391 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23788f2-dd90-4de4-b1e4-c2b424e60bf4 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.338951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23788f2-dd90-4de4-b1e4-c2b424e60bf4 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.339071 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23788f2-dd90-4de4-b1e4-c2b424e60bf4 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:18.340765 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 550/1073] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:18.347100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.347447 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.347668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:18.371067 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:18.372308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:18.397456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:18.459733 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 524/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:18.484088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23788f2-dd90-4de4-b1e4-c2b424e60bf4 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:18.544632 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:18.599988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23788f2-dd90-4de4-b1e4-c2b424e60bf4 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:18.641670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:18.642344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.642636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.642921 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:18.723433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:18.725706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:18.782917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:18.840437 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 551/1075] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:18.846144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:18.957695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d31b-38f2-4f4f-bd29-c7891d704f91 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:18.995939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:18.996692 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:18.996982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:18.997262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.007726 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:19.009742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-34e22a51-e12c-43c2-90f6-2ffa9b3d31d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:19.027833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:19.029314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:19.030741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:19.030741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:19.030741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:19.030741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:19.031302 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 525/1076] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:19.035436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:19.130743 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6a1a9-5005-464d-8068-533613391cc8 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.131326 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6a1a9-5005-464d-8068-533613391cc8 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.131326 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6a1a9-5005-464d-8068-533613391cc8 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.132985 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 526/1077] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:19.138580 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.138899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.139147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.151277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c38e9-a42c-4306-aad3-8fc0fa5fb66a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:19.156735 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:19.157938 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:19.160128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:19.191590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:19.192636 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.193069 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.193488 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.194637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:19.195114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:19.197369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:19.216254 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 552/1078] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:19.259940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 527/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:19.283799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:19.299049 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:19.395972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5440200-d3ab-4d7b-9c9d-31868ebd47f7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:19.434627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:19.435431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.435718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.436002 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.457012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:19.465484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:19.466520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:19.467440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 553/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:19.473032 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:19.596338 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e77e5-f747-42f0-b4f3-c50d8bbd6565 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:19.631834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:19.632637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.632831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.633109 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.634601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:19.634910 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:19.636305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:19.641506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:19.645611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 528/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:19.653130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:19.725672 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:19.767795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866017ff-94ed-480a-93b6-310c9cbc2e62 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:19.803042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:19.803920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.804212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.804526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] PATH_INFO: `/v3/users/ead082bbed5546e49dd9451966c0ccb8` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.805764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] RBAC: Authorizing `identity:get_user(user_id=ead082bbed5546e49dd9451966c0ccb8)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:19.807465 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:19.809151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 554/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:19 2022] GET /identity/v3/users/ead082bbed5546e49dd9451966c0ccb8 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:19.815096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:19.928852 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a897-5912-4c1e-b893-f3addf1dd95f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:19.963332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:19.964197 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:19.964513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:19.964792 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:19.965980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:19.967485 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6292a89-076c-4356-b53c-f686f26ddaf2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:19.969535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 529/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:19 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:19.975493 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:20.081419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f4751-8630-44bf-a2a8-ef37978c7262 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:20.116289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:20.117204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:20.117488 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:20.117766 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430/users/ead082bbed5546e49dd9451966c0ccb8/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:20.119816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] RBAC: Authorizing `identity:create_grant(project_id=a68bcd8a62b440169c14271779b85430, user_id=ead082bbed5546e49dd9451966c0ccb8, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:20.121285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9bcf4e-76c7-4bdb-ae79-f51d216c404d None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:20.130486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:20.137832 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 555/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:12:19 2022] PUT /identity/v3/projects/a68bcd8a62b440169c14271779b85430/users/ead082bbed5546e49dd9451966c0ccb8/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:12:20.677144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc3ee-e3f8-461d-b51d-6f65072c92ad None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:20.677687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc3ee-e3f8-461d-b51d-6f65072c92ad None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:20.677783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc3ee-e3f8-461d-b51d-6f65072c92ad None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:20.679107 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 530/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:20.685662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:20.686308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:20.686308 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:20.705257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:20.706495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:20.778360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 556/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:20.803081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dc3ee-e3f8-461d-b51d-6f65072c92ad None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:20.875234 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:20.913976 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dc3ee-e3f8-461d-b51d-6f65072c92ad None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:20.950458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:20.951198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:20.951487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:20.951764 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:20.964382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:20.972708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:20.973800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:20.974712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 531/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:12:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:20.980909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.090316 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcb06d-e9bc-42c4-9f8b-ba5c152e5394 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:21.117261 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:21.126534 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:21.127229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:21.127519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:21.127801 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:21.128600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:21.128917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:21.130304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:21.138871 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 557/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:12:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:21.144621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.259501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facd86ff-3302-419c-a02b-5978af09eb6f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:21.288768 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:21.297672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:21.298483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:21.298773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:21.299059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:21.310087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:21.311545 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:21.330470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:21.331612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:21.332806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:21.332806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:21.332806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:21.332806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:21.332806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 532/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:21.338559 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.444123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47ac80-a540-4245-9259-3b29184009e5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:21.478640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:21.479364 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:21.479652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:21.479933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:21.480738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:21.481035 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:21.482453 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:21.496150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 558/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:21.501816 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.605521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ca09a-b670-488d-9065-f2f6bff790f3 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:21.646371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:21.647172 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:21.647460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:21.647742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:21.655386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:21.656907 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:12:21.671468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:21.677380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:21.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 533/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:12:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:21.680030 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.732655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:21.785637 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd88a8f-819f-4e7b-9f54-4a0af9377ee8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:21.822560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:21.823242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:21.823533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:21.823817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:21.824696 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:21.825016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:21.826457 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:21.853640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 559/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:12:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:21.859460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:21.907916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:21.975042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0ffb8-0487-4caf-a68d-0b99f45c0ae3 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.011064 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.011731 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.012021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.012303 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.082342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.083834 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.127474 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:22.200137 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 534/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:22.206166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.326196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a4b1b-0411-48c2-b23c-a159d5be31e8 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.336664 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:22.367288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.368582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.369014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.369433 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.380435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.381836 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b644d00f-1f6c-4568-8764-e688f46811e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:22.400018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:22.401263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:22.402242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:12:22.402242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:22.402242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:12:22.402242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:22.402242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 560/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:22.406897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.497171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f41d-4ea3-4edd-a72d-16eb8a8425e0 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.497593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f41d-4ea3-4edd-a72d-16eb8a8425e0 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.497942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1f41d-4ea3-4edd-a72d-16eb8a8425e0 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.499401 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 561/1095] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:12:22.504370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.504715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.504995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.515522 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58d72-ffd6-4be1-a91e-943026c3e155 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.522669 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:22.523948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:22.551837 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.552546 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.552841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.553126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.553865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:22.554182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.555933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.564842 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:22.578211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 535/1096] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:22.584136 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.601574 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 562/1097] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:12:22.625429 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.695157 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae443c-52de-4df5-912b-eea5453489c2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.721712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:22.732047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.732969 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.733324 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.733686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.738917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaeeff-0a67-423b-90e7-e58ea1087072 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.740948 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.742429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:12:22.757256 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:22.758996 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:22.760085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 536/1098] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:22.764909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.776319 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.777543 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.777954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.778347 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.792209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:22.799947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:22.801159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:22.802568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 563/1099] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:12:22.807497 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.834012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:22.877973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42034c4-b3c7-423d-b367-3b33b985f19a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.914608 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508204f3-1233-4da4-bdfe-4b4baae12140 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:22.914884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.915697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.915994 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.916286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.917129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:22.917450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.918918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.931536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 537/1100] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:22.938325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.959374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:22.960003 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:22.960297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:22.960612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:22.961373 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:22.961726 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:22.963118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:22.971760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 564/1101] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:22.977540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:22.994043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:23.050153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efb1c8-2ac2-4aee-affb-84854c646d72 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:23.072093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:23.083420 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9175a8e-0e3d-426e-9e9d-a32ae7ef5460 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:23.088105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:23.088845 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:23.089131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:23.089406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:23.090263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] RBAC: Adding query filter params (role.id=adfaa919206d419c9f04614ea0f4d9ff, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=bdccfb4336b7491d8797c3f682ea0687) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:23.090579 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:23.092102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:23.098827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 538/1102] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/role_assignments?user.id=bdccfb4336b7491d8797c3f682ea0687&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=adfaa919206d419c9f04614ea0f4d9ff => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:23.120860 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:23.121769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:23.122056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:23.122339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] PATH_INFO: `/v3/users/ead082bbed5546e49dd9451966c0ccb8` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:23.123652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] RBAC: Authorizing `identity:get_user(user_id=ead082bbed5546e49dd9451966c0ccb8)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:23.125167 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:23.126927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 565/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:22 2022] GET /identity/v3/users/ead082bbed5546e49dd9451966c0ccb8 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:23.133001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:23.262422 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535245-c250-4df3-b85b-748aaa73ddf4 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:23.300643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:23.301326 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:23.301615 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:23.301897 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:23.359076 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:23.376489 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:23.377983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:23.485471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 539/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:23.491207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:23.591585 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:23.611900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc03e2-824e-4503-8f18-05375518ad13 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:23.648930 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:23.650083 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:23.650369 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:23.650649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:23.651953 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:23.653443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f287d-78fc-4dc7-9a52-85d8f16ad96c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:23.655174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 566/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:23 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:23.661375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:23.758380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:23.776160 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf55d-31af-43a8-89eb-bfe421b70d1f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:23.812463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:23.813126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:23.813412 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:23.813695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:23.814543 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a68bcd8a62b440169c14271779b85430, user.id=ead082bbed5546e49dd9451966c0ccb8) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:23.814857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:23.816313 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443f27-aa60-4252-ad65-cb46425eea29 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:23.824215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 540/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:23 2022] GET /identity/v3/role_assignments?user.id=ead082bbed5546e49dd9451966c0ccb8&scope.project.id=a68bcd8a62b440169c14271779b85430&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:26.133463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab036ec-77d6-49ff-a3ce-494985d3e2a1 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:26.134040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab036ec-77d6-49ff-a3ce-494985d3e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:26.134040 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab036ec-77d6-49ff-a3ce-494985d3e2a1 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:26.135388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 567/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:26.140913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:26.141360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:26.141468 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:26.158729 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:26.160026 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:26.232390 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 541/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:26.256732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab036ec-77d6-49ff-a3ce-494985d3e2a1 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:26.370314 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab036ec-77d6-49ff-a3ce-494985d3e2a1 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:26.406899 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:26.407706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:26.407994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:26.408273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:26.409477 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:26.410927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e9fda-b8ee-418d-be3d-73dbbf624320 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:26.412597 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 568/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:26.418656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:26.499186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:26.520939 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed96d8c-e681-48f3-8c09-75500e3e6bea None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:26.564818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:26.565501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:26.565793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:26.566078 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:26.566803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:26.568262 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a55c5-67e5-47cc-bf36-ad7060832edf None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:26.598698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 542/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:12:26 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:28.959159 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e6797-fb26-4b85-9d9a-2c56caae0891 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:28.959792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e6797-fb26-4b85-9d9a-2c56caae0891 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:28.959792 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e6797-fb26-4b85-9d9a-2c56caae0891 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:28.961160 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 569/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:28.966441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:28.966870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:28.967018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:28.984746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:28.985975 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:29.061248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 543/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:29.085471 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e6797-fb26-4b85-9d9a-2c56caae0891 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:29.104390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:29.189518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e6797-fb26-4b85-9d9a-2c56caae0891 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:29.223617 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:29.224425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:29.224707 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:29.224981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] PATH_INFO: `/v3/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:29.226459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] RBAC: Authorizing `identity:get_role(role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:29.227892 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:29.229300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 570/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:29 2022] GET /identity/v3/roles/519d740b9c3d46f3902513905864fa5d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:29.235349 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:29.300857 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:29.340733 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86f730-97be-47ee-8fe4-8c73c59105c2 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:29.376431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:29.377310 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:29.377592 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:29.377863 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] PATH_INFO: `/v3/users/7d8ee14f1db541eaa73649efd2f342ff` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:29.390712 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] RBAC: Authorizing `identity:get_user(user_id=7d8ee14f1db541eaa73649efd2f342ff)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:29.392265 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:29.394077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 544/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:29 2022] GET /identity/v3/users/7d8ee14f1db541eaa73649efd2f342ff => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:29.400186 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:29.486349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:29.502560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14314dd-c36c-4c61-9996-9ff43b8305bc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:29.535756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:29.536415 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:29.536708 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:29.536983 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:29.607322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:29.608881 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:29.693486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:29.720786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 571/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:29.726666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:29.828997 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e00d1-6e17-41fc-91c2-17654c203b7b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:29.851678 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:29.863901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:29.864829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:29.865101 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:29.865380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:29.866616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:29.868116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ebf9ac-58d0-452e-942b-a07437a65bd4 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:29.870096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 545/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:29 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:29.876385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:29.985964 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62fb7-3b26-415e-9539-e40e36a440da None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:30.021485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:30.022425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:30.022834 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:30.023234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:30.024505 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=a68bcd8a62b440169c14271779b85430, user.id=7d8ee14f1db541eaa73649efd2f342ff) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:30.025763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:30.028199 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5acc50-5f51-4bc1-9a45-3b36deaf523c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:30.035136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 572/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:29 2022] GET /identity/v3/role_assignments?user.id=7d8ee14f1db541eaa73649efd2f342ff&scope.project.id=a68bcd8a62b440169c14271779b85430&role.id=519d740b9c3d46f3902513905864fa5d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:32.268116 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f66b46-0d33-4ad3-a4b9-75990f48e351 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.268691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f66b46-0d33-4ad3-a4b9-75990f48e351 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.268830 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f66b46-0d33-4ad3-a4b9-75990f48e351 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:32.270103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 546/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:32.276690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.277148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.277270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:32.293951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:32.295149 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:32.340706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:32.363903 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 573/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:32.387573 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f66b46-0d33-4ad3-a4b9-75990f48e351 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:32.491778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f66b46-0d33-4ad3-a4b9-75990f48e351 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:32.527139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:32.528019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.528309 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.528627 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] PATH_INFO: `/v3/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:32.530156 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] RBAC: Authorizing `identity:get_role(role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:32.531548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:32.532998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 547/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:32 2022] GET /identity/v3/roles/519d740b9c3d46f3902513905864fa5d => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:32.538816 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:32.650482 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e09427-5d47-49b7-8c1f-11c9be7b0546 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:32.694360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:32.695239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.695529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.695810 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] PATH_INFO: `/v3/users/7d8ee14f1db541eaa73649efd2f342ff` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:32.697627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] RBAC: Authorizing `identity:get_user(user_id=7d8ee14f1db541eaa73649efd2f342ff)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:32.699078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:32.700791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 574/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:32 2022] GET /identity/v3/users/7d8ee14f1db541eaa73649efd2f342ff => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:32.706714 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:32.767769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:32.810749 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b60273-6279-44f6-a9b8-47846a07250c None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:32.846462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:32.847333 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.847621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.847900 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:32.849131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:32.850583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb42f8-bf4c-405e-98aa-15029b2a5507 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:32.852313 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 548/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:32 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:32.859085 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:32.907424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:32.961538 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa154e63-3c52-4c06-b60d-0622484a197b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:32.996661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:32.997568 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:32.997847 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:32.998148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430/users/7d8ee14f1db541eaa73649efd2f342ff/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:33.000440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] RBAC: Authorizing `identity:create_grant(project_id=a68bcd8a62b440169c14271779b85430, user_id=7d8ee14f1db541eaa73649efd2f342ff, role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:33.001888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdf672-dca2-4d40-900d-a982952b2b88 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:33.015682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 575/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:12:32 2022] PUT /identity/v3/projects/a68bcd8a62b440169c14271779b85430/users/7d8ee14f1db541eaa73649efd2f342ff/roles/519d740b9c3d46f3902513905864fa5d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:12:35.280835 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca1b0-fed0-42b7-8bc3-9c6f6ef99be3 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:35.281425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca1b0-fed0-42b7-8bc3-9c6f6ef99be3 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:35.281425 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca1b0-fed0-42b7-8bc3-9c6f6ef99be3 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:35.282810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 549/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:35.289034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:35.289588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:35.289588 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:35.306209 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:35.307656 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:35.375986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 576/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:35.402372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca1b0-fed0-42b7-8bc3-9c6f6ef99be3 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:35.505577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca1b0-fed0-42b7-8bc3-9c6f6ef99be3 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:35.531876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:35.541244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:35.542115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:35.542403 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:35.542681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] PATH_INFO: `/v3/roles/519d740b9c3d46f3902513905864fa5d` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:35.544229 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] RBAC: Authorizing `identity:get_role(role_id=519d740b9c3d46f3902513905864fa5d)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:35.545680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:35.547139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 550/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:35 2022] GET /identity/v3/roles/519d740b9c3d46f3902513905864fa5d => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:35.552996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:35.588076 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:35.656674 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb520197-5a4e-4099-a3c6-5e7fc186c905 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:35.691879 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:35.692796 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:35.693088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:35.693370 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] PATH_INFO: `/v3/users/7d8ee14f1db541eaa73649efd2f342ff` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:35.694599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] RBAC: Authorizing `identity:get_user(user_id=7d8ee14f1db541eaa73649efd2f342ff)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:35.697972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:35.700781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 577/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:35 2022] GET /identity/v3/users/7d8ee14f1db541eaa73649efd2f342ff => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:35.706903 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:35.812114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5095b7-c357-4931-ad78-e6e937beb042 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:35.847089 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:35.847737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:35.848025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:35.848308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:35.910010 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:35.918280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:35.919847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:36.025876 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 551/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:36.032279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:36.139243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc7b80-9b46-47e0-8c04-efb1b09260cb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:36.141633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:36.175648 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:36.176558 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:36.176842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:36.177114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] PATH_INFO: `/v3/projects/a68bcd8a62b440169c14271779b85430` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:36.178318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] RBAC: Authorizing `identity:get_project(project_id=a68bcd8a62b440169c14271779b85430)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:36.179770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ae535-984a-4892-8700-1568f0a40307 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:36.182135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 578/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:36 2022] GET /identity/v3/projects/a68bcd8a62b440169c14271779b85430 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:36.188675 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:36.283364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:36.290865 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d98a7-865a-409d-ad40-d22311053383 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:36.358591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:36.359254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:36.359547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:36.359833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:36.360707 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] RBAC: Adding query filter params (role.id=519d740b9c3d46f3902513905864fa5d, scope.project.id=a68bcd8a62b440169c14271779b85430, user.id=7d8ee14f1db541eaa73649efd2f342ff) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:36.361030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:36.362493 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255136-0e17-430b-b3b6-1e7080c35c03 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:36.368541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 552/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:36 2022] GET /identity/v3/role_assignments?user.id=7d8ee14f1db541eaa73649efd2f342ff&scope.project.id=a68bcd8a62b440169c14271779b85430&role.id=519d740b9c3d46f3902513905864fa5d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:38.685984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8828ca8-d0ee-457a-94ab-b752204e1b99 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:38.686606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8828ca8-d0ee-457a-94ab-b752204e1b99 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:38.686606 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8828ca8-d0ee-457a-94ab-b752204e1b99 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:38.687972 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 579/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:38.693963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:38.694452 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:38.694561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:38.711978 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:38.713331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:38.781983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 553/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:38.805501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8828ca8-d0ee-457a-94ab-b752204e1b99 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:38.911592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8828ca8-d0ee-457a-94ab-b752204e1b99 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:38.946900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:38.947767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:38.948056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:38.948340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:38.949764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:38.951219 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc55dfe-c32d-4f42-abb4-30879bdea2c0 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:38.952934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 580/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:38.958841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:39.049382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:39.062366 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73225e3-8c9c-4621-aeeb-47c21641e2b6 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:39.097941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:39.098638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:39.098918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:39.099206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:39.099960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:39.101515 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4bcd9-f2a9-44e7-8e64-c910c7af1d4b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:39.128114 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 554/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:12:38 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:41.411639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd492d-353b-4265-b286-c7a4a467e0d6 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:41.412202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd492d-353b-4265-b286-c7a4a467e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:41.412202 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd492d-353b-4265-b286-c7a4a467e0d6 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:41.413609 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 581/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:41.419419 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:41.419829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:41.419994 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:41.438047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:41.439282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:41.509383 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 555/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:41.534080 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd492d-353b-4265-b286-c7a4a467e0d6 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:41.588930 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:41.658096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd492d-353b-4265-b286-c7a4a467e0d6 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:41.694162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:41.694952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:41.695243 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:41.695525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:41.696882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:41.698242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afae0dc-8de0-445c-86d9-0889af8060ad None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:41.699990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 582/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:12:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:41.706204 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:41.777932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:41.811719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242957c-bec8-4376-bea6-9a6e9946911a None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:41.847833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:41.848549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:41.848847 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:41.849134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:41.849869 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:41.851349 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9185f-de33-4dc2-ac52-63388a9f061b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:41.875949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 556/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:12:41 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:12:44.273565 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f1f6-6f6f-436b-8178-2aaef84bfc3d None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:44.274156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f1f6-6f6f-436b-8178-2aaef84bfc3d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:44.274156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036f1f6-6f6f-436b-8178-2aaef84bfc3d None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:44.276115 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 583/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:44.281586 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:44.281991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:44.282163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:44.300211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:44.301516 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:44.375139 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 557/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:44.398187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f1f6-6f6f-436b-8178-2aaef84bfc3d None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:44.502950 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036f1f6-6f6f-436b-8178-2aaef84bfc3d None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:44.513242 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:44.539022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:44.539817 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:44.540103 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:44.541252 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:44.551462 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:44.560166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:44.561557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:44.562717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 584/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:44.568030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:44.625365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:44.672167 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c1d6-abd1-468b-82a2-6d6bb3a5b7a9 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:44.707251 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:44.707926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:44.708215 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:44.708528 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:44.709295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:44.709608 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:44.710998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:44.730200 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 558/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:44.736212 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:44.840926 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3c0f5-e3e8-4c99-ad93-d5b23ed2b4af None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:44.882528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:44.883428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:44.883717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:44.884012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] PATH_INFO: `/v3/users/8b23aa5088244387a3623e19e2bac496` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:44.898836 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] RBAC: Authorizing `identity:get_user(user_id=8b23aa5088244387a3623e19e2bac496)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:44.900640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:44.902344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 585/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:44 2022] GET /identity/v3/users/8b23aa5088244387a3623e19e2bac496 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:44.908185 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:45.013538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5ca92-41f0-4424-88fc-72a5332f9ccc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:45.039618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:45.058438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:45.059093 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:45.059377 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:45.059653 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:45.131198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:45.132658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:45.234440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 559/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:45.240630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:45.268578 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:45.343253 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d887c-42f8-4df5-bd63-168d5603ded4 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:45.378481 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:45.379348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:45.379635 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:45.379913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] PATH_INFO: `/v3/projects/49db9de4109842a9b1c539dfcd059b42` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:45.381123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] RBAC: Authorizing `identity:get_project(project_id=49db9de4109842a9b1c539dfcd059b42)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:45.382622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40cf21-87ff-4e74-9fc4-77a0caa7202f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:45.384353 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 586/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:45 2022] GET /identity/v3/projects/49db9de4109842a9b1c539dfcd059b42 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:45.390613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:45.410059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:45.493658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54410-b179-4019-86a3-51cb0acfa361 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:45.529710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:45.530378 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:45.530665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:45.530945 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:45.531787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=49db9de4109842a9b1c539dfcd059b42, user.id=8b23aa5088244387a3623e19e2bac496) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:45.532108 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:45.533611 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5e1c7-acdf-43cc-b042-c9e7d92d9b25 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:45.539031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 560/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:45 2022] GET /identity/v3/role_assignments?user.id=8b23aa5088244387a3623e19e2bac496&scope.project.id=49db9de4109842a9b1c539dfcd059b42&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:12:47.802204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43802bd3-0a4e-48c1-8a5b-217ff1676be4 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:47.802769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43802bd3-0a4e-48c1-8a5b-217ff1676be4 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:47.802769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43802bd3-0a4e-48c1-8a5b-217ff1676be4 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:47.804118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 587/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:47.810199 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:47.810618 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:47.810787 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:47.827922 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:47.829152 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:47.897819 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 561/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:47.920102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43802bd3-0a4e-48c1-8a5b-217ff1676be4 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:48.026114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:48.034333 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43802bd3-0a4e-48c1-8a5b-217ff1676be4 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:48.079318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:48.080034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:48.080320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:48.080627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:48.091101 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:48.098934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:48.100232 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:48.101345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 588/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:48.107507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:48.122319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:48.222813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ab21-e94a-460c-b224-b944df8085cb None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:48.264656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:48.265481 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:48.265637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:48.265918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:48.266695 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:48.267115 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:48.268432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:48.277401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 562/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:48.283343 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:48.388096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da66b0-e560-4f46-81cd-ce9f027adb95 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:48.424655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:48.425546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:48.425835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:48.426118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] PATH_INFO: `/v3/users/8b23aa5088244387a3623e19e2bac496` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:48.427348 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] RBAC: Authorizing `identity:get_user(user_id=8b23aa5088244387a3623e19e2bac496)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:48.428843 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:48.430518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 589/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:48 2022] GET /identity/v3/users/8b23aa5088244387a3623e19e2bac496 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:48.436761 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:48.514533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:48.541225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd94d4-d0f9-4c94-b6fa-a1c796cd0cc5 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:48.576186 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:48.577118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:48.577407 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:48.577690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] PATH_INFO: `/v3/projects/49db9de4109842a9b1c539dfcd059b42` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:48.579011 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] RBAC: Authorizing `identity:get_project(project_id=49db9de4109842a9b1c539dfcd059b42)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:48.580521 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb919d70-e1ea-44e7-a5b0-c4f048499118 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:48.582296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 563/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:48 2022] GET /identity/v3/projects/49db9de4109842a9b1c539dfcd059b42 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:48.589099 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:48.614425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:48.704105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342cfe8-5cbc-41af-93f9-edac056af877 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:48.739745 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:48.740673 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:48.740976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:48.741240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] PATH_INFO: `/v3/projects/49db9de4109842a9b1c539dfcd059b42/users/8b23aa5088244387a3623e19e2bac496/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:48.743273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=49db9de4109842a9b1c539dfcd059b42, user_id=8b23aa5088244387a3623e19e2bac496, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:48.745111 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03fed-ea3b-41e6-a87d-1610f35da0b2 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:48.758020 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 590/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:12:48 2022] PUT /identity/v3/projects/49db9de4109842a9b1c539dfcd059b42/users/8b23aa5088244387a3623e19e2bac496/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:12:51.112503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54ca79-8107-4d97-a96e-0f9473e844ce None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.113059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54ca79-8107-4d97-a96e-0f9473e844ce None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.113059 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54ca79-8107-4d97-a96e-0f9473e844ce None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.114427 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 564/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:51.121434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.121961 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.122235 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.141915 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:51.143119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:51.214424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 591/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:51.238356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54ca79-8107-4d97-a96e-0f9473e844ce None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:51.343599 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54ca79-8107-4d97-a96e-0f9473e844ce None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:51.377469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:51.378205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.378491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.378769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.390896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:12:51.398654 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:51.400342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:12:51.401477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 565/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:51.406668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:51.421333 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:51.508884 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa7492b-4d94-4f5a-9726-0a7db1163477 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:51.542996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:51.543677 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.543960 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.544234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.545033 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:51.545340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:51.546721 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:51.555462 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 592/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:51.561734 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:51.564651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:51.666084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85ac1c-133c-4d1e-94d3-a347b9ea7899 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:51.703047 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:51.703933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.704221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.704537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] PATH_INFO: `/v3/users/8b23aa5088244387a3623e19e2bac496` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.705754 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=8b23aa5088244387a3623e19e2bac496)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:51.707219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:51.708921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 566/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:12:51 2022] GET /identity/v3/users/8b23aa5088244387a3623e19e2bac496 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:51.715545 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:51.813698 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:51.839935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab923fe-a8bb-474e-8011-4fc3d2b859ca None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:51.878039 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:51.878698 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:51.878988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:51.879279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:51.948898 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:51.950282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:52.029372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:52.062818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 593/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:52.069342 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:52.143161 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:52.175492 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f4e03-9526-4517-ac4d-1df8828ca9c2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:52.213726 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:52.214530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:52.214818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:52.215102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] PATH_INFO: `/v3/projects/49db9de4109842a9b1c539dfcd059b42` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:52.216414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] RBAC: Authorizing `identity:get_project(project_id=49db9de4109842a9b1c539dfcd059b42)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:52.217920 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5455ba-6af3-4a6e-b228-0fa2fb20cdd5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:52.219711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 567/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:12:52 2022] GET /identity/v3/projects/49db9de4109842a9b1c539dfcd059b42 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:52.225920 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:52.330520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4eaf-685e-403d-8ea8-84f2e3f3944c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:52.366023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:52.366728 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:52.367415 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:52.367994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:52.369874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=49db9de4109842a9b1c539dfcd059b42, user.id=8b23aa5088244387a3623e19e2bac496) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:52.370328 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:52.372532 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff07f-97b1-4a27-9a0e-ce50ff4bd007 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:52.381151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 594/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:12:52 2022] GET /identity/v3/role_assignments?user.id=8b23aa5088244387a3623e19e2bac496&scope.project.id=49db9de4109842a9b1c539dfcd059b42&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:54.663548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae86999-3825-4b2b-8f9d-fbe16703517c None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:54.664245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae86999-3825-4b2b-8f9d-fbe16703517c None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:54.664245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae86999-3825-4b2b-8f9d-fbe16703517c None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:54.666580 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 568/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:54.672742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:54.673311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:54.673311 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:54.691172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:54.692354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:54.723239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:54.763935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 595/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:54.787216 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae86999-3825-4b2b-8f9d-fbe16703517c None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:54.891253 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae86999-3825-4b2b-8f9d-fbe16703517c None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:54.925356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:54.925944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:54.926230 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:54.926513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:54.936189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:54.997358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:54.998824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:55.102856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 569/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:12:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:12:55.108980 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:55.221294 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76cc3f-26ef-420c-ba69-dd78cbb5679e None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:55.265518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:55.266318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:55.266603 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:55.266880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:55.273593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:55.275978 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-cb381058-e6ad-418b-9941-64ad8e6bd38a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:12:55.291485 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:12:55.293059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:12:55.294355 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 596/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:12:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:12:55.299352 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:55.338812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:55.418064 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef25ea-c2e7-4c6e-bb17-912fa331cca1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:55.452824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:55.453428 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:55.453719 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:55.454006 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:55.454756 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:12:55.455074 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:55.456500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8555ec92-a7cd-4178-8256-4f9c47c914f9 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:55.464829 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 570/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:12:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:12:57.730136 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847627a-1832-47f1-aab7-ab86bb15050c None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:57.730719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847627a-1832-47f1-aab7-ab86bb15050c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:57.730719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847627a-1832-47f1-aab7-ab86bb15050c None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:57.732095 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 597/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:12:57.737788 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b48b8-d25c-4ebe-a053-2e5286865fce None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:57.738280 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b48b8-d25c-4ebe-a053-2e5286865fce None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:57.738385 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b48b8-d25c-4ebe-a053-2e5286865fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:57.761590 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-806b48b8-d25c-4ebe-a053-2e5286865fce None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:12:57.762826 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-806b48b8-d25c-4ebe-a053-2e5286865fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:12:57.838325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 571/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:12:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:12:57.862838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847627a-1832-47f1-aab7-ab86bb15050c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:12:57.893116 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:12:57.980723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847627a-1832-47f1-aab7-ab86bb15050c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:12:58.017948 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:12:58.018659 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:12:58.018952 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:12:58.019237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:12:58.020016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:12:58.021440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d84ee3-b3a4-41fe-b5da-80667a768fcc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:12:58.037774 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 598/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu May 19 20:12:57 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:13:00.335195 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e367-30c7-4856-9e3e-82c5c2ab1e54 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:00.335728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e367-30c7-4856-9e3e-82c5c2ab1e54 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:00.335862 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3e367-30c7-4856-9e3e-82c5c2ab1e54 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:00.337751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 572/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:13:00.342967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458801a-4033-4b66-8742-81862955d178 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:00.343358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458801a-4033-4b66-8742-81862955d178 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:00.343536 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458801a-4033-4b66-8742-81862955d178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:00.361976 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-c458801a-4033-4b66-8742-81862955d178 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:13:00.363182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-c458801a-4033-4b66-8742-81862955d178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:13:00.431076 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 599/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:13:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:13:00.454687 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e367-30c7-4856-9e3e-82c5c2ab1e54 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:00.488914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:00.566748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3e367-30c7-4856-9e3e-82c5c2ab1e54 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:00.603221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:00.604077 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:00.604364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:00.604741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] PATH_INFO: `/v3/domains/eb792635a45f4433bf7ae91bc4852acc` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:00.615563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb792635a45f4433bf7ae91bc4852acc)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:00.617187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65811fa3-9d24-456f-9848-2d2d4ee5b325 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:00.619110 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 573/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 20:13:00 2022] GET /identity/v3/domains/eb792635a45f4433bf7ae91bc4852acc => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:00.625038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458801a-4033-4b66-8742-81862955d178 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:00.643471 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:00.728682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c458801a-4033-4b66-8742-81862955d178 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:00.762361 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:00.763025 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:00.763320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:00.763595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:00.764304 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:00.765783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15eb954-465d-4e63-9477-e148f888c3c9 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:00.793371 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 600/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:13:00 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:13:03.168211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8d28-6bfb-44e2-abe2-b47e0c367b91 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:03.168810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8d28-6bfb-44e2-abe2-b47e0c367b91 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:03.168810 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b8d28-6bfb-44e2-abe2-b47e0c367b91 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:03.170239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 574/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:13:03.176183 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:03.176695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:03.176795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:03.194824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:13:03.196105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:13:03.260554 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:03.266041 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 601/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:13:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:13:03.290933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b8d28-6bfb-44e2-abe2-b47e0c367b91 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:03.395911 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:03.406475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b8d28-6bfb-44e2-abe2-b47e0c367b91 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:03.441944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:03.442753 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:03.443046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:03.443325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] PATH_INFO: `/v3/domains/eb792635a45f4433bf7ae91bc4852acc` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:03.444554 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=eb792635a45f4433bf7ae91bc4852acc)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:03.446058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e39724-2d54-4133-a3ba-db2dd2ca2aab None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:03.447906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 575/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 20:13:03 2022] GET /identity/v3/domains/eb792635a45f4433bf7ae91bc4852acc => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:03.453657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:03.556010 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a7120-5576-4ae8-9880-188c2829ff2c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:03.591877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:03.592586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:03.592876 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:03.593158 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:03.593893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:03.595339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35697-9d58-4302-8b0c-a066b6f53efa None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:03.615946 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 602/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:13:03 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:13:05.905410 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680187e-d085-4941-888e-f8a5bcc85eb9 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:05.905984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680187e-d085-4941-888e-f8a5bcc85eb9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:05.905984 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680187e-d085-4941-888e-f8a5bcc85eb9 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:05.907367 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 576/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:13:05.912790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:05.913182 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:05.913356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:05.939881 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:13:05.941132 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:13:05.994187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:06.011436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 603/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:13:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:13:06.044244 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680187e-d085-4941-888e-f8a5bcc85eb9 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:06.150968 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680187e-d085-4941-888e-f8a5bcc85eb9 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:06.185721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:06.186559 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:06.186856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:06.187144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:06.199818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:13:06.207665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:13:06.208917 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:13:06.210140 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 577/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:13:06.214967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:06.320071 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198cfbf5-977c-4ad1-b96e-f284c000f445 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:06.358237 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:06.358909 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:06.359195 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:06.359470 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:06.360227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:13:06.360572 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:06.361945 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:06.370340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 604/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:06.376085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:06.398404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:06.480681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d09d5-78e5-4622-b372-2f58773497bc None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:06.518901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:06.519784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:06.520073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:06.520356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] PATH_INFO: `/v3/users/f03131420e7f4059b92ff905130c534a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:06.534118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] RBAC: Authorizing `identity:get_user(user_id=f03131420e7f4059b92ff905130c534a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:06.535583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:06.537308 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 578/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:13:06 2022] GET /identity/v3/users/f03131420e7f4059b92ff905130c534a => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:06.544196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:06.603581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:06.654480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e51d-68fb-461e-ad2b-85b4d8b2affd None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:06.689690 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:06.690343 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:06.690630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:06.690906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:06.762576 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:06.763955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:06.812316 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:06.864521 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 605/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:13:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:13:06.870958 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:06.951328 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:06.974789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e1e69-59d0-406a-a631-68972c5a7a6e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:07.010063 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:07.010926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:07.011220 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:07.011503 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] PATH_INFO: `/v3/projects/6509c17b256a4ef1a7cba6d98b021601` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:07.012784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] RBAC: Authorizing `identity:get_project(project_id=6509c17b256a4ef1a7cba6d98b021601)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:07.014196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182968b-5408-403f-aad1-985c88d0f627 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:07.025270 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 579/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:13:06 2022] GET /identity/v3/projects/6509c17b256a4ef1a7cba6d98b021601 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:07.031448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:07.136137 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd1738-8991-4bc4-93b2-e24699889290 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:07.172454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:07.173133 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:07.173427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:07.173713 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:07.174554 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=6509c17b256a4ef1a7cba6d98b021601, user.id=f03131420e7f4059b92ff905130c534a) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:13:07.174868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:07.176367 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8052935-6634-49dd-8b6c-daf7549683ea None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:07.183368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 606/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:13:07 2022] GET /identity/v3/role_assignments?user.id=f03131420e7f4059b92ff905130c534a&scope.project.id=6509c17b256a4ef1a7cba6d98b021601&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:09.433264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5172b4-621a-4925-a2a3-fe6fdb946bf3 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:09.433822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5172b4-621a-4925-a2a3-fe6fdb946bf3 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:09.433822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5172b4-621a-4925-a2a3-fe6fdb946bf3 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:09.435196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 580/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:13:09.440668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:09.441106 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:09.441240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:09.457993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:13:09.459102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:13:09.467719 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:09.529735 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 607/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:13:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:13:09.553226 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5172b4-621a-4925-a2a3-fe6fdb946bf3 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:09.666159 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5172b4-621a-4925-a2a3-fe6fdb946bf3 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:09.709913 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:09.710623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:09.710911 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:09.711191 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:09.722069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:13:09.731042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:13:09.732501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:13:09.733867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 581/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:13:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:13:09.738459 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:09.844129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec837fa5-bba4-4412-8717-414a74f091ee None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:09.878770 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:09.879491 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:09.879789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:09.880058 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:09.880866 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:13:09.881177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:09.882582 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:09.893811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 608/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:09.900549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:09.910073 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:10.016372 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5452e2b-964e-4082-af7d-3f9cdf5edd36 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:10.054394 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:10.055311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:10.055668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:10.056177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] PATH_INFO: `/v3/users/f03131420e7f4059b92ff905130c534a` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:10.057439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] RBAC: Authorizing `identity:get_user(user_id=f03131420e7f4059b92ff905130c534a)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:10.059144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:10.060833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 582/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:13:09 2022] GET /identity/v3/users/f03131420e7f4059b92ff905130c534a => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:10.066590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:10.070426 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:10.169758 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd752df1-9be8-4259-9b36-867a403095e5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:10.211110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:10.211988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:10.212273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:10.212586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] PATH_INFO: `/v3/projects/6509c17b256a4ef1a7cba6d98b021601` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:10.213781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] RBAC: Authorizing `identity:get_project(project_id=6509c17b256a4ef1a7cba6d98b021601)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:10.215221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b9a8c-4aa5-4b38-8d29-9cb1ca50ff75 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:10.216990 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 609/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:13:10 2022] GET /identity/v3/projects/6509c17b256a4ef1a7cba6d98b021601 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:10.222893 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:10.313284 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:10.339205 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb220b4c-d3b9-4df4-a431-8bf810a1d490 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:10.374296 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:10.375194 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:10.375483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:10.375766 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] PATH_INFO: `/v3/projects/6509c17b256a4ef1a7cba6d98b021601/users/f03131420e7f4059b92ff905130c534a/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:10.377873 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] RBAC: Authorizing `identity:create_grant(project_id=6509c17b256a4ef1a7cba6d98b021601, user_id=f03131420e7f4059b92ff905130c534a, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:10.379332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c2c22-3788-4473-99b1-90720b8b99df None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:10.392821 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 583/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:13:10 2022] PUT /identity/v3/projects/6509c17b256a4ef1a7cba6d98b021601/users/f03131420e7f4059b92ff905130c534a/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:13:12.760281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5a47-0827-4959-b8b6-94c2114c1fca None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:12.760891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5a47-0827-4959-b8b6-94c2114c1fca None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:12.760891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca5a47-0827-4959-b8b6-94c2114c1fca None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:12.762249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 610/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:13:12.767568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:12.767982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:12.768148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:12.785399 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:13:12.786616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:13:12.860182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 584/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:13:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:13:12.883064 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca5a47-0827-4959-b8b6-94c2114c1fca None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:12.958652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:12.990463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca5a47-0827-4959-b8b6-94c2114c1fca None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.026045 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.026984 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.027285 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.027573 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.038470 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:13:13.046266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:13:13.047443 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:13:13.048595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 611/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:13:13.061595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:13.125798 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:13.166406 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17664c-af9d-4b83-b8dd-667add88b9e5 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.205752 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.206432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.214624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.214624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.214624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:13:13.214624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:13.214624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:13.219263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 585/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:13:13.225100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:13.326414 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a51356-650f-4cfe-b14a-4b8dc82225df None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.356109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:13.361724 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.362615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.362900 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.363184 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] PATH_INFO: `/v3/users/f03131420e7f4059b92ff905130c534a` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.364518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] RBAC: Authorizing `identity:get_user(user_id=f03131420e7f4059b92ff905130c534a)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:13.365977 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:13.367703 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 612/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:13:13 2022] GET /identity/v3/users/f03131420e7f4059b92ff905130c534a => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:13.373643 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:13.476460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786794b-fdeb-4f1f-a189-9b95005f63bd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.506589 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:13.516463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.517128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.517417 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.517698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.591806 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:13.593219 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:13.695185 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 586/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:13:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:13:13.701757 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:13.806784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08471049-2dba-4c00-98df-7fde1e47432c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.844222 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.845052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.845340 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.845614 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] PATH_INFO: `/v3/projects/6509c17b256a4ef1a7cba6d98b021601` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.846835 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] RBAC: Authorizing `identity:get_project(project_id=6509c17b256a4ef1a7cba6d98b021601)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:13.848283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91df96-f999-4ede-a55c-b093af1c8b7b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:13.850060 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 613/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:13:13 2022] GET /identity/v3/projects/6509c17b256a4ef1a7cba6d98b021601 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:13.856460 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:13:13.883963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:13.960214 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b890a73c-bf09-4f38-a468-5174085785fb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:13:13.995331 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:13:13.996022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:13:13.996319 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:13:13.996766 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:13:13.997645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=6509c17b256a4ef1a7cba6d98b021601, user.id=f03131420e7f4059b92ff905130c534a) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:13:13.997965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:13:13.999470 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1528a-a7ca-4d27-b86a-a990aa6ad839 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:13:14.006085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 587/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:13:13 2022] GET /identity/v3/role_assignments?user.id=f03131420e7f4059b92ff905130c534a&scope.project.id=6509c17b256a4ef1a7cba6d98b021601&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:13:23.839578 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=66 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:13:24.003144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=55 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:14.724381 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5c921-d873-4eab-afce-a084dff836c0 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:14.725027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5c921-d873-4eab-afce-a084dff836c0 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:14.725027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5c921-d873-4eab-afce-a084dff836c0 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:14.728634 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 614/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:14.732282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:14.732783 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:14.732875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:14.757566 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:14.760232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:14.910021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 588/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:14.932444 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5c921-d873-4eab-afce-a084dff836c0 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:15.069535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5c921-d873-4eab-afce-a084dff836c0 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:15.110761 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:15.111577 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:15.111858 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:15.112138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:15.131122 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:15.132995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:15.152466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:15.153625 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:15.154763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:15.154763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:15.154763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:15.154763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:15.154763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 615/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:15.163813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:15.336896 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481c17-1220-4675-b7df-c5f3397a030d None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:15.380748 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:15.390286 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:15.391449 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:15.394501 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:15.394944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:15.396450 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:15.396582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:15.398101 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb51ebf-53f9-47e9-83f3-98395bb7f82b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:15.420437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 589/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:15.421732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:15.508456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:15.579902 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221583df-bea2-4a9f-91c6-14229a9cee22 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:15.633612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:15.634424 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:15.634727 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:15.634997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:15.635739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:15.637276 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a3521b-0010-42a0-a70d-51934d9b3c00 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:15.662805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 616/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:14:15 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:14:18.197105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2316b8-9975-4523-81e3-77bdba0cb692 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:18.197596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2316b8-9975-4523-81e3-77bdba0cb692 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:18.197690 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2316b8-9975-4523-81e3-77bdba0cb692 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:18.199058 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 590/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:18.205509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:18.206151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:18.206473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:18.226140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:18.228027 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:18.311434 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 617/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:18.345207 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2316b8-9975-4523-81e3-77bdba0cb692 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:18.491758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2316b8-9975-4523-81e3-77bdba0cb692 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:18.535794 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:18.536699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:18.536990 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:18.539849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:18.553745 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:18.563840 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:18.565123 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:18.566051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:18.569737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 591/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:14:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:18.573014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:18.624384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:18.701289 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9afd1f-cf72-4a33-b02f-1d7804e61733 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:18.752300 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:18.753947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:18.754622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:18.755404 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:18.757585 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:18.758718 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:18.761005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:18.771310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 618/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:14:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:18.778600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:18.853552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:18.915320 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bff7fe-56b6-4fdc-bd67-cd85a329a29d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:18.958901 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:18.959812 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:18.960148 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:18.960495 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:18.973196 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:18.975131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:18.996778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:18.997891 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:18.998888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:18.998888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:18.998888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:18.998888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:18.998888 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 592/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:19.005633 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:19.185612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:19.191605 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b7005-115e-4281-86d9-09b672450030 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:19.246038 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:19.246874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:19.247169 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:19.247469 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:19.248456 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:19.248760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:19.250519 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:19.266783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 619/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:19.272991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:19.401741 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55f069-9748-456a-9d59-cbbb3298d502 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:19.403389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:19.454046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:19.455022 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:19.455274 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:19.455559 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:19.464666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:19.466172 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:19.488595 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:19.495694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:19.497021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 593/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:19.499267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:19.635861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fccf3-6c91-4eb9-b0aa-f283deb2a501 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:19.675667 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:19.676484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:19.676874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:19.677132 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:19.678117 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:19.678425 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:19.680863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:19.702454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 620/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:19.707970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:19.855769 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2a30a-53b1-44ab-ae21-590bc0c02c64 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:19.907001 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:19.907671 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:19.907965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:19.908263 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:19.952430 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:20.000905 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:20.002391 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:20.137563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 594/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:14:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:14:20.144093 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:20.166431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:20.282539 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f82de-e907-47f8-95cd-2b90c42a9c86 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:20.329120 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:20.329934 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:20.330222 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:20.330502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:20.349392 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:20.351192 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:20.373542 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:20.375123 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:20.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:20.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:20.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:20.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:20.376134 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 621/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:20.381588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:20.458211 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:20.514375 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc7465-1917-4cae-ac6c-607de5ae687d None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:20.563667 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:20.564823 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:20.565341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:20.565813 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:20.566943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:20.567402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:20.569454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:20.596736 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 595/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:20.599140 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:20.692153 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:20.730883 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15190f08-0a15-42d1-95e2-9d71782b73d3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:20.784955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:20.785842 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:20.786249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:20.786566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:20.796110 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:20.797288 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:20.814864 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:20.816007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:20.823389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 622/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:20.826225 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:20.970822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41942a3a-bfa5-46f4-a4df-3342bcb3bfa3 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:20.998992 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:21.016416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:21.017239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:21.017535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:21.017902 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:21.024623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:21.024623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:21.024623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b6c09a-0b15-47c7-a0ad-7e05a2f143f2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:21.041318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 596/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:21.047537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:21.173297 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6436-b46d-423c-8da7-93abdfafd4c7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:21.220867 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:21.229882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:21.230560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:21.230856 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:21.231157 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:21.232019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=b12622da600d4f88955999f8c33863c7) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:21.232359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:21.233901 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60f50e-5cec-47f7-a792-b2bd2e9e75a4 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:21.241265 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 623/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:14:21 2022] GET /identity/v3/role_assignments?user.id=b12622da600d4f88955999f8c33863c7&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:23.669232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117dc7-ba1f-433b-8e96-2cdf7068afeb None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:23.669844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117dc7-ba1f-433b-8e96-2cdf7068afeb None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:23.669844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89117dc7-ba1f-433b-8e96-2cdf7068afeb None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:23.671754 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 597/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:23.677856 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:23.678479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:23.678479 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:23.700778 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:23.702105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:23.804599 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 624/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:23.822691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89117dc7-ba1f-433b-8e96-2cdf7068afeb None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:23.952024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89117dc7-ba1f-433b-8e96-2cdf7068afeb None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:23.989698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:23.990506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:23.990791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:23.991070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.003360 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:24.011999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:24.013085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:24.014018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 598/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:24.019266 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:24.136062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eae90-3d4a-4088-9070-2934f654ce15 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:24.166451 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:24.178714 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:24.179372 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:24.179661 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:24.179956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.180868 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] RBAC: Adding query filter params (name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:24.181188 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:24.182714 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:24.192198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 625/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:24.198568 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:24.246184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:24.319918 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4989c8f-9b7a-443f-b880-7d1ed0ad21fd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:24.367491 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:24.368661 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:24.368943 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:24.369254 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.383644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:24.385792 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:24.406233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:24.407294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:24.408299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:24.408299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:24.408299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:24.408299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:24.408299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 599/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:24.414198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:24.535166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb4b5c-c335-4940-9469-bd8f42802477 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:24.578262 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:24.578993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:24.579338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:24.579739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.580626 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:24.580955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:24.582513 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:24.598118 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 626/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:24.604658 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:24.728860 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:24.737257 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50114e-1465-4de7-90f0-91a2d12fc6c6 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:24.779926 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:24.780954 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:24.781463 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:24.781681 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.791584 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:24.793330 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:24.810685 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:24.812025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:24.813283 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 600/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:24.818253 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:24.871354 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:24.938148 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69597a-de7f-4278-ad49-a66ac2381170 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:24.981503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:24.982200 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:24.982487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:24.982765 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:24.983647 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:24.983967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:24.985587 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.006333 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 627/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:25.012317 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:25.138345 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c87d8-3f7a-4365-b804-02203f6407c1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:25.176180 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:25.177015 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:25.177304 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:25.177592 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:25.190850 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:25.192420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:25.213956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:25.215233 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:25.216373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:25.216373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:25.216373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:25.216373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:25.216373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 601/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:25.221179 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:25.339345 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf58b9-368c-4afe-847c-e4ee70b56c2c None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:25.341922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:25.381050 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:25.381784 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:25.382066 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:25.382344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:25.383119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:25.383428 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:25.385100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.398388 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 628/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:25.404674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:25.415112 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:25.518680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a11e8-04a1-41ae-bafb-88007b6fd879 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:25.555081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:25.556037 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:25.556306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:25.556688 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:25.565349 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:25.566982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:25.582531 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:25.584057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:25.585293 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:25.588711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 602/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:25.591279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:25.712929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245909ab-4559-4257-8c0c-cadf0e15c8da None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:25.750511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:25.751299 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:25.751502 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:25.751769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:25.752639 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:25.752912 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:25.754362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04026a21-847d-4d85-9ccb-91d7e4de8e55 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.769695 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 629/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:25.776631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:25.867684 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:25.907028 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3014-1f7a-4a7c-8e42-cdb9968f6719 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:25.957402 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:25.958316 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] REQUEST_METHOD: `PUT` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:25.958648 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:25.958906 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/b12622da600d4f88955999f8c33863c7/roles/86e3ecd166d04281bf8d64df34c21b06` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:25.975423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=b12622da600d4f88955999f8c33863c7, role_id=86e3ecd166d04281bf8d64df34c21b06)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:25.977506 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbf71d-c921-4d0d-b48e-ea1ca3f475e5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:25.991773 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 603/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:14:25 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/b12622da600d4f88955999f8c33863c7/roles/86e3ecd166d04281bf8d64df34c21b06 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:14:28.575804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1edb50-56e5-4467-9dd4-62480ad8c712 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:28.576334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1edb50-56e5-4467-9dd4-62480ad8c712 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:28.576334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1edb50-56e5-4467-9dd4-62480ad8c712 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:28.577729 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 630/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:28.584772 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:28.585243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:28.585369 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:28.603383 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:28.604785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:28.687548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 604/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:28.716994 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1edb50-56e5-4467-9dd4-62480ad8c712 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:28.744682 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:28.856799 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1edb50-56e5-4467-9dd4-62480ad8c712 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:28.895414 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:28.896238 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:28.896560 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:28.896839 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] PATH_INFO: `/v3/roles/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:28.907715 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:28.916790 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:28.918112 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:28.919229 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 631/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:28.925051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:28.983177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:29.044003 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923addff-9155-48cf-bf2d-fbca8f207748 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:29.088183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:29.088899 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:29.089188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:29.089469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:29.090237 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:29.090549 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:29.091936 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:29.102027 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 605/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:29.108208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:29.208630 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:29.225642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3193e40-c80c-44b2-9f6e-6eafc0d93caf None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:29.261557 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:29.262356 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:29.262640 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:29.262916 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:29.273663 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:29.275422 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:29.294848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:29.296023 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:29.297226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:29.297226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:29.297226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:29.297226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:29.297226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 632/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:29.302046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:29.416098 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89047-56ed-4a26-b058-bf506ada1fc2 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:29.426496 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:29.457224 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:29.458083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:29.460201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:29.460201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:29.460201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:29.460201 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:29.461771 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:29.476174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 606/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:29.484096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:29.597657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4781f8-cde9-4e0b-aa51-b803f1677e27 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:29.635362 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:29.636176 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:29.636500 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:29.636785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:29.640881 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:29.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:29.649967 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:29.667812 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:29.670380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:29.672988 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:29.675374 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 633/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:29.675584 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:29.810541 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29adc8-e6aa-4a52-8eb5-0a559ef334bd None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:29.848102 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:29.848827 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:29.849117 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:29.849400 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:29.850305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:29.850622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:29.852965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:29.873673 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 607/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:29.879717 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:29.995173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd4ff68-b630-447a-9362-c4a4860746cc None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:30.043504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:30.044394 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:30.044806 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:30.045245 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:30.117129 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:30.125387 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:30.127015 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:30.249303 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 634/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:14:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:14:30.255507 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:30.289612 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:30.403552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10c985-a470-4c42-a5ce-7e58b3e9312e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:30.452483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:30.453321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:30.453638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:30.453995 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:30.465069 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:30.466674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:30.488856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:30.495570 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:30.496793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:30.496793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:30.496793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:30.496793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:30.496793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 608/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:30.497487 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:30.609125 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:30.616702 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98d22-4f62-4d78-9191-2f47ebe2b7b1 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:30.660324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:30.660973 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:30.661393 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:30.661782 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:30.662699 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:30.663018 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:30.664716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:30.683913 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 635/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:30.687624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:30.812733 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:30.842577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e6cca-4ade-4ea9-bc40-b29e0a0115fb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:30.884831 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:30.885759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:30.886018 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:30.894802 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:30.903605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:30.905416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:30.920188 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:30.921433 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:30.922437 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 609/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:30.927555 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:31.047922 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c0e2f-8903-47d1-9dc7-680b07ffb3ba None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:31.108436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:31.109598 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:31.109941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:31.110346 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:31.111380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:31.111739 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:31.113815 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85b33f-73f4-4cf4-aed2-be31abf47eeb None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:31.130850 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 636/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:31.140318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:31.280266 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d969d98-9582-4e27-9995-345f335967e8 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:31.320900 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:31.321620 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:31.321892 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:31.322183 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:31.323134 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] RBAC: Adding query filter params (role.id=86e3ecd166d04281bf8d64df34c21b06, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=b12622da600d4f88955999f8c33863c7) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:31.323462 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:31.325053 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3ff11-b747-4a0a-802c-f241fe6a3eca None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:31.332356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 610/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:14:31 2022] GET /identity/v3/role_assignments?user.id=b12622da600d4f88955999f8c33863c7&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=86e3ecd166d04281bf8d64df34c21b06 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:33.716495 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb38e0-381d-4580-a7a3-4f69446cdf21 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:33.717130 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb38e0-381d-4580-a7a3-4f69446cdf21 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:33.717234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb38e0-381d-4580-a7a3-4f69446cdf21 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:33.718642 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 637/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:33.724710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:33.725239 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:33.725339 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:33.731409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:33.744591 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:33.745814 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:33.841875 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 611/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:33.865942 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb38e0-381d-4580-a7a3-4f69446cdf21 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:33.887961 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:33.988380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb38e0-381d-4580-a7a3-4f69446cdf21 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:34.033462 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:34.034273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:34.034559 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:34.034837 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:34.047771 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:34.056807 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:34.058014 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:34.059607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 638/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:14:34.064858 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:34.178356 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b8886-61f8-4e09-88bf-ea72fb692edf None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:34.226044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:34.233072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:34.233820 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:34.234081 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:34.234400 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:34.235234 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:34.235562 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:34.237044 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:34.249118 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 612/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:34.255224 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:34.355359 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:34.383660 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618e4b9-c005-493b-b37e-27cf7a06cacb None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:34.429291 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:34.430794 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:34.431217 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:34.431413 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:34.444283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:34.446048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:34.467072 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:34.468349 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:34.469533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:34.469533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:34.469533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:34.469533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:34.475012 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 639/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:34.476030 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:34.607964 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3da15b-7c09-4a0d-91a8-2d27dc97b148 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:34.646387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:34.647104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:34.647401 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:34.647694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:34.649314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:34.649889 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:34.651553 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:34.676206 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 613/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:34.682756 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:34.802277 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cdec9-06ea-4717-b2ca-03c270d083f7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:34.843065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:34.852829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:34.853686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:34.853979 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:34.854259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] PATH_INFO: `/v3/users/placement` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:34.864874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:34.866943 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:34.886927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:34.888358 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:34.889468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 640/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:34.896854 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:34.963744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:35.017576 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8097f-27c3-4cb7-9647-8c175a286f5e None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:35.062355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:35.063174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:35.063476 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:35.063759 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:35.065024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:35.065358 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:35.067867 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:35.092537 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 614/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:35.098805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:35.268509 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80712f79-bf50-4694-bdae-9de7b6a2716b None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:35.310658 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:35.311365 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:35.311656 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:35.311939 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:35.373056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:35.396881 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:35.399313 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:35.520081 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 641/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:14:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:14:35.521454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:35.634785 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bfd67-a779-4ba8-96dc-fb92ab953bdf None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:35.654704 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:35.688944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:35.689752 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:35.690051 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:35.690321 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:35.704380 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:35.706034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:35.730056 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:35.735404 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:35.736631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:35.736631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:35.736631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:35.736631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:35.736631 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 615/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:35.737580 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:35.864096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:35.878043 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51fb82-12d7-4fe4-9e63-f0b4f92dd710 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:35.937171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:35.937925 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:35.938263 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:35.938591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:35.939395 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:35.939714 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:35.941292 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:35.958670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 642/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:35.965602 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:36.088414 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605a9c6-32b5-4d19-8d61-2ff40b6d1d52 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:36.127774 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:36.128603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:36.128914 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:36.129221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:36.137679 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:36.139142 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:36.154622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:36.155970 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:36.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 616/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:36.162019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:36.310483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91c7df-751d-4027-ab0e-7aeb817c478a None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:36.351114 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:36.352529 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:36.352828 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:36.353022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] PATH_INFO: `/v3/projects` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:36.353827 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:36.354151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:36.355781 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f49eb-bec9-4111-b20f-97e70f4eb26b None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:36.368389 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 643/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:36.374689 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:36.404409 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:36.501155 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15398b-4bd9-4d16-90db-0dcf407d7685 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:36.538279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:36.539699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:36.540104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:36.540710 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:36.541623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=b12622da600d4f88955999f8c33863c7) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:36.541995 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:36.543513 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850774e6-d8d5-421a-b342-0409df25e162 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:36.557336 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 617/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:14:36 2022] GET /identity/v3/role_assignments?user.id=b12622da600d4f88955999f8c33863c7&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:38.881135 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8582-33d1-446c-8601-863173cbd44f None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:38.881805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8582-33d1-446c-8601-863173cbd44f None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:38.881805 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c8582-33d1-446c-8601-863173cbd44f None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:38.883818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 644/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:38.890084 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:38.890644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:38.890644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:38.908088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:38.909309 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:38.983478 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 618/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:14:39.006701 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8582-33d1-446c-8601-863173cbd44f None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.018357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:39.111007 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c8582-33d1-446c-8601-863173cbd44f None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:39.146065 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:39.146886 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:39.147172 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:39.147449 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:39.159151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:39.167432 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:39.168822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:39.170053 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 645/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:39.174713 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.194582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:39.279838 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0959dc8-e9a0-4570-8a53-e6fcb7dc0eec None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:39.319432 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:39.320844 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:39.321290 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:39.321583 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] PATH_INFO: `/v3/roles` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:39.322348 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:39.322656 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:39.324039 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:39.332721 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 619/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:39.339139 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.417484 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:39.451594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b0fb2-3037-4a7b-9874-51195e0ff2a7 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:39.488793 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:39.489598 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:39.489882 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:39.490163 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:39.501462 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:39.511365 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:39.529956 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:39.531161 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:39.532584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:39.532584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:39.532584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:39.532584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:39.532584 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 646/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:39.537049 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.620870 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:39.640441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726daa36-aa4c-4f01-8621-2e27bb839887 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:39.676535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:39.677131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:39.677418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:39.677699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:39.678435 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:39.678746 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:39.680959 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:39.693292 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 620/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:39.699150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.807174 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca5da0-916c-495a-8109-8fa9ae320a09 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:39.832749 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:39.848788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:39.849594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:39.849877 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:39.850151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] PATH_INFO: `/v3/users/placement` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:39.858365 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:39.859804 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:39.874177 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:39.875600 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:39.876893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 647/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:39.881458 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:39.984452 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b646b48-caa9-4a4a-94c2-9a22c6a52b76 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.018231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.018898 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.019178 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.019456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] PATH_INFO: `/v3/users` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.020294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:40.020638 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.022061 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.032213 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:40.042038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 621/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:14:40.047738 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:40.156166 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853cdb-0a92-46e4-816a-07109b69bf11 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.201207 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.202019 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.202306 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.202592 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.214156 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.215612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:40.232689 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:40.234366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:40.235622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:40.235622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:40.235622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:40.235622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:40.235622 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 648/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:40.239933 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:40.346487 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d85b58-e28b-47fc-99a4-a2f95ae1ac50 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.381042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.381716 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.381999 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.382279 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.383577 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:40.383886 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.385294 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.397565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 622/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:40.403995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:40.435189 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:40.507085 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf3a42-27bf-4644-bc5c-23fccf4dcf99 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.541258 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.542048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.542332 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.542610 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.549951 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.551416 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:40.566971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:40.568227 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:40.569324 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 649/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:40.574696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:40.636264 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:40.680822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e85a8b-99cb-489e-adb0-88fd7fbe377a None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.716379 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.717091 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.717382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.717666 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.718471 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:40.718789 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.720243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ddfbd-fe06-421c-b0bc-618b1f3dc4d5 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.732641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 623/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:40.738878 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:40.839795 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:40.848803 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb9411-5666-4192-bbfc-417e115891f3 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:40.892282 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:40.893206 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] REQUEST_METHOD: `PUT` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:40.893501 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:40.893783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] PATH_INFO: `/v3/projects/a5b0478445f54f2286808b9731547668/users/b12622da600d4f88955999f8c33863c7/roles/557c5948d33a461a89e131c3fc5aca4e` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:40.895786 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] RBAC: Authorizing `identity:create_grant(project_id=a5b0478445f54f2286808b9731547668, user_id=b12622da600d4f88955999f8c33863c7, role_id=557c5948d33a461a89e131c3fc5aca4e)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:40.897281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76a1f9-6798-4149-9a0c-80a3b9cf4802 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:40.910047 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 650/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:14:40 2022] PUT /identity/v3/projects/a5b0478445f54f2286808b9731547668/users/b12622da600d4f88955999f8c33863c7/roles/557c5948d33a461a89e131c3fc5aca4e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:14:43.141008 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec1d-a2b4-4659-9ff2-0ffbb84eb6f3 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.141603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec1d-a2b4-4659-9ff2-0ffbb84eb6f3 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.141603 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaec1d-a2b4-4659-9ff2-0ffbb84eb6f3 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.142965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 624/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:43.148318 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.148760 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.148880 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.167195 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:43.168450 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:43.241581 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 651/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:43.265606 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaec1d-a2b4-4659-9ff2-0ffbb84eb6f3 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:43.371231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaec1d-a2b4-4659-9ff2-0ffbb84eb6f3 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:43.389504 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:43.407864 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:43.408623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.408915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.409198 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.421214 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:14:43.429054 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:43.430442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:43.431796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 625/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:14:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:43.436971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:43.525380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:43.545657 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf71ae-c859-4274-bc23-62f4c2493aff None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:43.583463 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:43.584142 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.584467 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.584767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.585526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:43.585997 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:43.587504 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:43.595789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 652/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:14:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:43.601644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:43.715935 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29842-501c-4c08-9ca7-702f10155ae1 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:43.751696 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:43.752530 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.752822 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.753105 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.767803 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:43.769552 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:43.788438 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:43.789921 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:43.791572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:43.791572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:43.791572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:43.791572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:43.791572 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 626/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:43.795985 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:43.911998 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f825bb2-d6f3-4cc9-9d96-686819921ae5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:43.949036 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:43.949706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:43.949993 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:43.950270 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] PATH_INFO: `/v3/domains` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:43.951004 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:43.951320 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:43.952716 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:43.965256 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 653/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:43.971046 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:44.003980 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:44.076016 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1544e6-ef98-4763-8b02-85a8735fdafb None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:44.110376 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:44.111184 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:44.111467 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:44.111742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] PATH_INFO: `/v3/users/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:44.119940 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:44.121416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:14:44.136021 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:44.137258 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:44.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 627/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:44.143257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:44.147341 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:44.247546 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a1ee3-bfb8-4f84-ad50-bfce33f5c6d5 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:44.282418 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:44.283096 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:44.283384 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:44.283666 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] PATH_INFO: `/v3/users` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:44.284551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:44.284869 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:44.286310 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:44.306181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 654/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:14:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:44.312248 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:44.408909 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:44.420282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cbff5-9d80-404c-b539-a000d8e4f1ca None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:44.465355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:44.466010 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:44.466299 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:44.466581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:44.537111 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:44.538477 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:44.644318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:44.655575 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 628/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:14:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:14:44.661819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:44.724955 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:44.767063 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c5ebe-e3fa-4960-99a2-e7c304527710 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:44.804269 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:44.805612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:44.805895 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:44.806168 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:44.816767 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:44.818234 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:14:44.837226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:44.838480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:44.839711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:14:44.839711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:14:44.839711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:14:44.839711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:44.839711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 655/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:44.844345 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:44.953581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb26e4-6935-432c-ae9b-56b09944bb7f None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:44.989699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:44.990373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:44.990659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:44.990938 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] PATH_INFO: `/v3/domains` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:44.991670 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:44.992062 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:44.993613 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:45.004974 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 629/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:45.010773 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:45.111783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c994e-5df4-4dcb-a3fe-c594e107334e None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:45.125665 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:45.147535 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:45.148960 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:45.149241 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:45.149607 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:45.156352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:45.157811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:14:45.180380 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application with Lock( May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:45.181679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: ERROR keystone.server.flask.application May 19 20:14:45.182723 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 656/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:45.191092 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:45.243895 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:45.297916 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb0be6-8d02-483a-a298-e8297d43a37b None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:45.331983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:45.332679 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:45.332963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:45.333243 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] PATH_INFO: `/v3/projects` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:45.334038 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:45.334355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:45.337440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1351-c059-4dce-b496-a9e7e94fcc1c None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:45.354176 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 630/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:14:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:45.360226 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:45.470198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca1947-186e-4565-9252-3e495de70c23 None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:45.512162 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:45.512861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:45.513152 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:45.513435 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:45.514775 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] RBAC: Adding query filter params (role.id=557c5948d33a461a89e131c3fc5aca4e, scope.project.id=a5b0478445f54f2286808b9731547668, user.id=b12622da600d4f88955999f8c33863c7) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:45.515097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:45.516590 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e94cf-c13b-463a-8333-6cf4d07eeaab None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:45.522594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 657/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:14:45 2022] GET /identity/v3/role_assignments?user.id=b12622da600d4f88955999f8c33863c7&scope.project.id=a5b0478445f54f2286808b9731547668&role.id=557c5948d33a461a89e131c3fc5aca4e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:47.778016 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c56a-dd2c-4622-8875-05739e1acc73 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:47.778601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c56a-dd2c-4622-8875-05739e1acc73 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:47.778601 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4c56a-dd2c-4622-8875-05739e1acc73 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:47.780088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 631/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:47.786789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:47.787360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:47.787468 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:47.808533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:47.809998 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:47.847928 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:47.886982 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 658/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:47.910181 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c56a-dd2c-4622-8875-05739e1acc73 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:48.016221 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4c56a-dd2c-4622-8875-05739e1acc73 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:48.026973 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:48.051817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:48.052567 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:48.052856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:48.053135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:48.053809 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:48.056042 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:48.068343 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:48.069728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:48.071596 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 632/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:14:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:48.076510 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:48.181791 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef27202-b8cc-4c04-a26f-69f2fcfac98e None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:48.218239 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:48.218914 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:48.219204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:48.219471 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:48.220175 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] RBAC: Adding query filter params (name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:48.220524 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:48.221894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b92936-80a4-4f74-a07f-90f56e63d83f None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:48.226915 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 659/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:14:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:48.232686 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:48.335705 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6e52-e48a-4e08-a03b-f2af138c96b0 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:48.370793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:48.371469 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:48.371751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:48.372031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] PATH_INFO: `/v3/services` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:48.372778 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:48.373090 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:48.374475 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483960ea-c179-43c5-86e5-6eca5dcaecc1 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:48.378952 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 633/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:14:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:50.718021 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745272e-c295-408b-afae-da7712f1bcfa None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:50.718620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745272e-c295-408b-afae-da7712f1bcfa None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:50.718620 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745272e-c295-408b-afae-da7712f1bcfa None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:50.719954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 660/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:50.725341 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8242-1b7d-4ab9-b443-471804b36c3d None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:50.725800 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8242-1b7d-4ab9-b443-471804b36c3d None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:50.725907 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e8242-1b7d-4ab9-b443-471804b36c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:50.742538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-2c6e8242-1b7d-4ab9-b443-471804b36c3d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:50.743723 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-2c6e8242-1b7d-4ab9-b443-471804b36c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:50.781423 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:50.822781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 634/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:50.846198 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745272e-c295-408b-afae-da7712f1bcfa None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:50.877971 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:50.951929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745272e-c295-408b-afae-da7712f1bcfa None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:50.994516 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:50.995105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:50.995395 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:50.995679 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:50.996454 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:50.997838 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b2d06-125b-4945-8bde-143f1a4b5fa7 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:51.012190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 661/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:14:50 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:14:53.358377 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613c74-fcad-42ab-a51e-cc9d74e66922 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:53.358965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613c74-fcad-42ab-a51e-cc9d74e66922 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:53.358965 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1613c74-fcad-42ab-a51e-cc9d74e66922 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:53.360355 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 635/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:53.366048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:53.366466 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:53.366615 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:53.384427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:53.385628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:53.459566 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 662/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:53.482100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613c74-fcad-42ab-a51e-cc9d74e66922 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:53.587718 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1613c74-fcad-42ab-a51e-cc9d74e66922 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:53.624641 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:53.625451 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:53.625742 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:53.626026 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:53.626697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:53.628096 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:53.640145 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:53.641588 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:53.642963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 636/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:14:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:53.647700 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:53.668173 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:53.760675 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8ec9e-0e29-48e6-a737-092f1b27386c None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:53.804915 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:53.805595 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:53.805888 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:53.806171 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:53.806890 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] RBAC: Adding query filter params (name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:53.807204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:53.808855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:53.814105 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 663/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:14:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:53.820100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:53.853310 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:53.931963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8246d-9f9e-4c3d-8565-a7e3dabc35a3 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:53.976013 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:53.977276 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:53.977566 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:53.977849 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:53.982080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] RBAC: Adding query filter params (interface=public, service_id=7154a28092674284af6f582e8443500a, region_id=RegionOne) {{(pid=214149) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:53.982456 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:53.983941 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3c9ca-e267-4a91-b5d5-1c98d13fa111 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:53.990373 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 637/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:14:53 2022] GET /identity/v3/endpoints?service_id=7154a28092674284af6f582e8443500a&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:53.996246 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:54.080740 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:54.100873 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f899a94-6d28-46ac-b9e5-7896be2873ea None admin] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:54.135193 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:54.135905 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:54.136196 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:54.136506 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:54.137208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:54.138612 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de43a7-69d0-4e2c-824b-46c7e9284447 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:54.146181 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 664/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:14:53 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:14:56.393442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a244-22a2-44bb-960c-c97d0ba3cbec None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:56.394012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a244-22a2-44bb-960c-c97d0ba3cbec None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:56.394012 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a244-22a2-44bb-960c-c97d0ba3cbec None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:56.395416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 638/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:14:56.401208 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:56.401644 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:56.401863 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:56.420187 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:14:56.421483 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:14:56.490551 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 665/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:14:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:14:56.514672 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a244-22a2-44bb-960c-c97d0ba3cbec None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:56.615621 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:56.623699 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a244-22a2-44bb-960c-c97d0ba3cbec None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:56.658535 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:56.659347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:56.659833 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:56.659963 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] PATH_INFO: `/v3/services/placement` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:56.660600 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:56.661992 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:14:56.674659 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:14:56.675987 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application with Lock( May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self._enter() May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application return self.creator() May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application created_value = creator( May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: ERROR keystone.server.flask.application May 19 20:14:56.677189 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 639/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:14:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:14:56.682811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:56.763074 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:14:56.786398 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861402a-44b6-43c1-9c66-b39a9da39d31 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:56.822279 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:56.822954 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:56.823236 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:56.823511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] PATH_INFO: `/v3/services` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:56.824222 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=214148) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:14:56.824554 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:56.825925 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27b717-dd82-414a-ba12-74fe403a10f6 None admin] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:56.831005 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 666/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:14:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:14:56.837072 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:14:56.942758 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8750d-4d0d-4bbd-956a-f242725f3627 None admin] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:14:56.978154 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfca2eff03f04a12a3f7eb5950753314', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:14:56.978818 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:14:56.979103 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:14:56.979383 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:14:56.980128 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:14:56.981539 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0f359-5380-4963-88f6-9fe4171e7dbf None admin] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:14:57.001124 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 640/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:14:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:15:06.835876 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=30 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:15:06.996664 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:06.110419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d7a7-cc40-455f-ba02-612d2311f64b None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:06.111008 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d7a7-cc40-455f-ba02-612d2311f64b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:06.111097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145d7a7-cc40-455f-ba02-612d2311f64b None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:06.112533 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 667/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:06.121024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923c6c-ec4d-44cb-88f1-8a03f532c5d9 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:06.121441 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923c6c-ec4d-44cb-88f1-8a03f532c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:06.121728 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b923c6c-ec4d-44cb-88f1-8a03f532c5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:06.252318 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-8b923c6c-ec4d-44cb-88f1-8a03f532c5d9 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:06.253793 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-8b923c6c-ec4d-44cb-88f1-8a03f532c5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:06.538622 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 641/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:06.674503 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:06.674861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:06.675144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:06.676678 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 668/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 20:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:06.691738 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d9c5-3036-4512-8ee6-c0187272a260 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:06.692070 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d9c5-3036-4512-8ee6-c0187272a260 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:06.692314 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af82d9c5-3036-4512-8ee6-c0187272a260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:06.858418 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-af82d9c5-3036-4512-8ee6-c0187272a260 None None] MFA Rules not processed for user `2aa1864ac1e1481a9c09f418e9978944`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:06.859825 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-af82d9c5-3036-4512-8ee6-c0187272a260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:06.909841 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:07.029143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 642/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:07.042249 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:07.044476 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:07.238429 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2940c-e89f-4348-b3f6-382d39e9f1ba None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:07.385278 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:07.386100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:07.386518 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:07.386906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:07.388924 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:07.468856 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:07.565861 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:07.567499 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98548087-c702-43aa-a9b4-54e0c3c4d840 service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:07.594788 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 669/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:10.529131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:10.529635 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:10.529635 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:10.535182 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 643/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:10.536947 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3a234-8354-4496-92fe-e5cbcacc0d02 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:10.537845 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3a234-8354-4496-92fe-e5cbcacc0d02 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:10.538352 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3a234-8354-4496-92fe-e5cbcacc0d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:10.634436 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-6bf3a234-8354-4496-92fe-e5cbcacc0d02 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:10.635824 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-6bf3a234-8354-4496-92fe-e5cbcacc0d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:10.759455 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 670/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:16:10.802363 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:10.804744 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:10.940068 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44389521-f54e-4ab0-9d17-944651b9b7c9 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:11.027249 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:11.030791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:11.031551 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:11.031856 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:11.032143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:11.035007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:11.173998 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:11.175565 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4141-53c8-4a0f-9e90-556078c73053 service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:11.201932 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 644/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:13.878375 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:13.878974 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:13.878974 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:13.880336 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 671/1315] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:13.886533 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738c70-c366-4772-ba2b-5688f6cee8fc None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:13.886947 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738c70-c366-4772-ba2b-5688f6cee8fc None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:13.887104 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de738c70-c366-4772-ba2b-5688f6cee8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:13.963192 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-de738c70-c366-4772-ba2b-5688f6cee8fc None None] MFA Rules not processed for user `a7c5be151f744680932039d47817a89c`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:13.964446 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-de738c70-c366-4772-ba2b-5688f6cee8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:14.018958 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:14.065637 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 645/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:14.106514 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:14.108078 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:14.122729 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:14.230473 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d744250-7ba9-4e31-87b0-0a5b6fe22003 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:14.306772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:14.307421 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:14.307706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:14.307981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:14.309906 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:14.375150 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:14.425052 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:14.426511 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0b9d4-716d-41e3-986d-4905b2f15c85 service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:14.448593 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 672/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:17.403131 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:17.403660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:17.403660 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:17.405085 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 646/1318] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:17.413325 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc724-d00c-4696-9685-62632b9c8b4d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:17.414144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc724-d00c-4696-9685-62632b9c8b4d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:17.414144 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0dc724-d00c-4696-9685-62632b9c8b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:17.485073 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-ec0dc724-d00c-4696-9685-62632b9c8b4d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:17.486283 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-ec0dc724-d00c-4696-9685-62632b9c8b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:17.585917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 673/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:17.626240 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:17.627791 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:17.750143 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08597d-eca3-4d15-85eb-e1b925ccd1b6 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:17.828698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:17.829361 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:17.829651 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:17.829934 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:17.831623 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:17.839255 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:17.945669 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:17.947144 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba539a-a5b0-43ae-8d51-6e44c54ce86a service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:17.966751 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 647/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:21.051796 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:21.052377 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:21.052377 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:21.054929 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 674/1321] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:21.061781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cf50-053b-4df8-a8a4-6456ded36720 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:21.062282 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cf50-053b-4df8-a8a4-6456ded36720 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:21.062387 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476cf50-053b-4df8-a8a4-6456ded36720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:21.130275 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-0476cf50-053b-4df8-a8a4-6456ded36720 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:21.131500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-0476cf50-053b-4df8-a8a4-6456ded36720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:21.202563 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:21.251756 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 648/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:16:21.288525 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:21.290092 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:21.311338 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:21.403406 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d9b263-f759-4996-b8b2-bc918a8c9aed None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:21.481471 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:21.482127 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:21.482412 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:21.482691 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:21.484339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:21.565686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:21.611789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:21.613379 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f44bcb-00c6-4d0b-b3f5-33e79e1591dc service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:21.638102 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 675/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:24.001164 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1bc9-838f-45dc-88ec-51659c571a0b None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:24.001711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1bc9-838f-45dc-88ec-51659c571a0b None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:24.001711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe1bc9-838f-45dc-88ec-51659c571a0b None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:24.003135 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 649/1324] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:16:24.009779 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4b4e-7d3c-436b-b311-48af91143ff3 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:24.010296 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4b4e-7d3c-436b-b311-48af91143ff3 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:24.010407 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea4b4e-7d3c-436b-b311-48af91143ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:24.080591 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-57ea4b4e-7d3c-436b-b311-48af91143ff3 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:24.081834 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-57ea4b4e-7d3c-436b-b311-48af91143ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:24.199741 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 676/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:24.261775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:24.262083 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:24.262442 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:24.263988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 650/1326] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 20:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:24.271244 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cdd-0ee6-4977-9b48-08231924e09c None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:24.271537 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cdd-0ee6-4977-9b48-08231924e09c None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:24.271848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cdd-0ee6-4977-9b48-08231924e09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:24.305180 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:24.353818 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-bf656cdd-0ee6-4977-9b48-08231924e09c None None] MFA Rules not processed for user `2aa1864ac1e1481a9c09f418e9978944`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:24.355042 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-bf656cdd-0ee6-4977-9b48-08231924e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:24.444527 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 677/1327] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 20:16:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:24.454187 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:24.455843 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:24.580960 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79344d4e-6b20-4339-8629-5861bb4118d6 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:24.661034 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:24.661698 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:24.661988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:24.662270 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:24.664928 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:24.685737 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:24.788122 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:24.789757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8ed76-3cc3-47dd-874c-00afc92d9515 service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:24.809516 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 651/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:27.452034 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:27.452586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:27.452586 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:27.453959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 678/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:27.459365 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fca994-03d0-4783-9ec1-6016b21bfc39 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:27.459915 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fca994-03d0-4783-9ec1-6016b21bfc39 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:27.460697 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fca994-03d0-4783-9ec1-6016b21bfc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:27.528153 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-e1fca994-03d0-4783-9ec1-6016b21bfc39 None None] MFA Rules not processed for user `a7c5be151f744680932039d47817a89c`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:27.529429 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-e1fca994-03d0-4783-9ec1-6016b21bfc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:27.603087 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:27.636080 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 652/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:27.674662 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:27.676221 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:27.784652 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc457f5-13dc-4b99-a2bb-629552c762fc None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:27.811100 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:27.867431 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:27.868097 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:27.868390 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:27.868706 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:27.870385 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:27.982829 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:27.984305 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39b7d9-89ea-424e-993b-4ff1feaaaebd service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:28.002427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 679/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:30.740561 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:30.741197 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:30.741197 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:30.742668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 653/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:30.747935 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b394f8-2930-4cfb-a2ef-ce1cdcaf10fa None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:30.748333 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b394f8-2930-4cfb-a2ef-ce1cdcaf10fa None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:30.748540 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b394f8-2930-4cfb-a2ef-ce1cdcaf10fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:30.814891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:30.818056 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-b3b394f8-2930-4cfb-a2ef-ce1cdcaf10fa None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:30.820138 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-b3b394f8-2930-4cfb-a2ef-ce1cdcaf10fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:30.921694 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 680/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:30.957624 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:30.959174 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:31.073347 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607764ba-173f-40cc-942a-01ee47ea46b9 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:31.152703 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:31.153379 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:31.153665 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:31.153942 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:31.155648 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:31.183088 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:31.281775 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:31.283232 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdf2df-bc6d-45a6-b581-59fd928dec5c service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:31.303382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 654/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:33.658855 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:33.659410 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:33.659410 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:33.660917 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 681/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:33.666743 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f1bf-3377-4862-96ef-5a41e6fe5459 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:33.667312 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f1bf-3377-4862-96ef-5a41e6fe5459 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:33.667312 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f1bf-3377-4862-96ef-5a41e6fe5459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:33.744031 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-ac06f1bf-3377-4862-96ef-5a41e6fe5459 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:33.745371 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-ac06f1bf-3377-4862-96ef-5a41e6fe5459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:33.825295 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:33.856881 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 655/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:33.900674 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:33.902240 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:34.009627 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4433c-ede7-4d0f-9cd1-cafdb78787d9 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:34.037936 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:34.087959 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:34.088649 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:34.088941 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:34.089223 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:34.090891 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:34.218737 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:34.220205 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba704e-efa9-4f16-81c6-a52c94b253fd service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:34.239025 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 682/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:36.888384 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:36.889057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:36.889057 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:36.890392 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 656/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:36.895785 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953143a-5db3-476f-99d9-fe5c7c93ad6d None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:36.896259 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953143a-5db3-476f-99d9-fe5c7c93ad6d None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:36.896368 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953143a-5db3-476f-99d9-fe5c7c93ad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:36.963697 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-5953143a-5db3-476f-99d9-fe5c7c93ad6d None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:36.965016 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-5953143a-5db3-476f-99d9-fe5c7c93ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:36.975022 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:37.080070 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 683/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:37.116411 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:37.118781 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:37.248014 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56d2b8-1958-416a-b570-a4e872cc69af None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:37.251739 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:37.252389 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:37.252708 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:37.252988 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:37.345784 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:37.347249 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1b8f9-3684-49e0-9d65-2ede8c75e7ca service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:37.366026 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 657/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:41.784121 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2d64f-685a-4f0d-a6ee-fa53e199d54a None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:41.784747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2d64f-685a-4f0d-a6ee-fa53e199d54a None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:41.784747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2d64f-685a-4f0d-a6ee-fa53e199d54a None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:41.786119 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 684/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:41.792100 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d200-92af-47c0-ac80-4702ec0622af None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:41.792581 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d200-92af-47c0-ac80-4702ec0622af None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:41.792694 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d200-92af-47c0-ac80-4702ec0622af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:41.802645 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=100 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:41.875177 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-8436d200-92af-47c0-ac80-4702ec0622af None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:41.876427 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-8436d200-92af-47c0-ac80-4702ec0622af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:41.968454 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 658/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:42.008334 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2d64f-685a-4f0d-a6ee-fa53e199d54a None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:42.020970 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2d64f-685a-4f0d-a6ee-fa53e199d54a None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:42.027109 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:42.028088 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:42.028554 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:42.028968 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:42.039769 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:42.041257 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ce895-9d02-4586-8299-599f5b591860 service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:42.050772 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 685/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:45.051440 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe419639-34bf-4209-b24b-d7e42f06f513 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:45.052007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe419639-34bf-4209-b24b-d7e42f06f513 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:45.052007 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe419639-34bf-4209-b24b-d7e42f06f513 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:45.054416 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 659/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:45.060062 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4509e-081e-4468-a36d-5a003815959b None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:45.060654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4509e-081e-4468-a36d-5a003815959b None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:45.060654 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4509e-081e-4468-a36d-5a003815959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:45.091423 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-02a4509e-081e-4468-a36d-5a003815959b None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:45.092668 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-02a4509e-081e-4468-a36d-5a003815959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:45.100969 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 686/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:45.138498 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe419639-34bf-4209-b24b-d7e42f06f513 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:45.149311 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe419639-34bf-4209-b24b-d7e42f06f513 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:45.152944 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:45.153593 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:45.153884 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:45.154170 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:45.164961 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:45.166500 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969463ca-9523-4b23-9ae2-d7e47b08ed7c service neutron] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:45.177883 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 660/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:49.588789 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef29a-174a-4589-ab4b-f246649514df None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:49.589344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef29a-174a-4589-ab4b-f246649514df None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:49.589344 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef29a-174a-4589-ab4b-f246649514df None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:49.590763 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 687/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:49.595991 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170e22-1fe7-4741-8147-ba986e935f36 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:49.596538 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170e22-1fe7-4741-8147-ba986e935f36 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:49.596668 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84170e22-1fe7-4741-8147-ba986e935f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:49.617125 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-84170e22-1fe7-4741-8147-ba986e935f36 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:49.618386 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-84170e22-1fe7-4741-8147-ba986e935f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:49.626532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 661/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:49.666747 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef29a-174a-4589-ab4b-f246649514df None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:16:49.678228 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef29a-174a-4589-ab4b-f246649514df None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:16:49.681894 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa1864ac1e1481a9c09f418e9978944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:16:49.682561 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:49.682850 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:49.683131 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:49.693616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:16:49.695079 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166ee4e-24e4-4afa-b241-7bac828beabe service neutron] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:16:49.705074 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 688/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:16:55.561332 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3940f0-29f2-47d7-8414-d8ca41fca758 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:55.563024 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3940f0-29f2-47d7-8414-d8ca41fca758 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:55.563483 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3940f0-29f2-47d7-8414-d8ca41fca758 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:55.565228 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 662/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:16:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:16:55.577322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a1db-aee4-4bb5-ba31-bb0c3d8d684e None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:16:55.578215 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a1db-aee4-4bb5-ba31-bb0c3d8d684e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:16:55.578656 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a1db-aee4-4bb5-ba31-bb0c3d8d684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:16:55.695874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-33b0a1db-aee4-4bb5-ba31-bb0c3d8d684e None None] MFA Rules not processed for user `b12622da600d4f88955999f8c33863c7`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:16:55.697486 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-33b0a1db-aee4-4bb5-ba31-bb0c3d8d684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:16:55.749731 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=100 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:16:55.790594 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 689/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:16:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:16:59.611757 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=56 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:00.191274 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39692c-c8bf-46e7-989a-dfd925fc53ef None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:00.191709 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39692c-c8bf-46e7-989a-dfd925fc53ef None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:00.191949 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39692c-c8bf-46e7-989a-dfd925fc53ef None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:00.195982 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 663/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:00.199628 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d2b-19d2-4bfd-860c-194470d0be60 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:00.200151 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d2b-19d2-4bfd-860c-194470d0be60 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:00.200265 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d2b-19d2-4bfd-860c-194470d0be60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:00.219490 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-8d926d2b-19d2-4bfd-860c-194470d0be60 None None] MFA Rules not processed for user `b12622da600d4f88955999f8c33863c7`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:00.220742 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-8d926d2b-19d2-4bfd-860c-194470d0be60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:00.230729 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 690/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:17:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:02.048129 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c4ad5-542f-4de4-94e1-8de12b961c5f None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:02.048817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c4ad5-542f-4de4-94e1-8de12b961c5f None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:02.048817 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c4ad5-542f-4de4-94e1-8de12b961c5f None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:02.050126 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 664/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:02.056011 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae4ef-6441-42e8-b707-c5379a5011f8 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:02.056520 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae4ef-6441-42e8-b707-c5379a5011f8 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:02.056658 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aae4ef-6441-42e8-b707-c5379a5011f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:02.076655 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-a7aae4ef-6441-42e8-b707-c5379a5011f8 None None] MFA Rules not processed for user `b12622da600d4f88955999f8c33863c7`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:02.077874 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-a7aae4ef-6441-42e8-b707-c5379a5011f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:02.087273 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 691/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:17:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:17:06.242388 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f7b4-1e72-4447-b3a3-8a9ca6762249 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:06.242986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f7b4-1e72-4447-b3a3-8a9ca6762249 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:06.242986 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f7b4-1e72-4447-b3a3-8a9ca6762249 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:06.244886 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 665/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:06.250705 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a26a-bb52-4119-a608-ca7b656322fe None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:06.251528 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a26a-bb52-4119-a608-ca7b656322fe None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:06.251981 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57a26a-bb52-4119-a608-ca7b656322fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:06.275267 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-ca57a26a-bb52-4119-a608-ca7b656322fe None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:06.276872 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-ca57a26a-bb52-4119-a608-ca7b656322fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:06.292366 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 692/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:06.352542 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54bf6-b939-47f0-9f63-b73874690e83 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:06.353413 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54bf6-b939-47f0-9f63-b73874690e83 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:06.354305 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d54bf6-b939-47f0-9f63-b73874690e83 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:06.357674 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 666/1358] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:17:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:06.366448 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2866c3-d353-49d2-8bc4-9bcda132f559 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:06.366933 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2866c3-d353-49d2-8bc4-9bcda132f559 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:06.367322 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2866c3-d353-49d2-8bc4-9bcda132f559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:06.427191 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-cb2866c3-d353-49d2-8bc4-9bcda132f559 None None] MFA Rules not processed for user `bdccfb4336b7491d8797c3f682ea0687`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:06.428737 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-cb2866c3-d353-49d2-8bc4-9bcda132f559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:06.517944 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 693/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:17:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:06.526872 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d54bf6-b939-47f0-9f63-b73874690e83 None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:06.542066 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d54bf6-b939-47f0-9f63-b73874690e83 None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:06.548163 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdccfb4336b7491d8797c3f682ea0687', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:06.549147 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:06.549439 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:06.549727 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:06.559711 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:06.561364 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64bdd-c263-42ac-8d9a-33bd843063b6 service nova] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:06.572025 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 667/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:17:09.106397 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28953001-0269-474b-a1c1-d1c06719e3f5 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:09.106927 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28953001-0269-474b-a1c1-d1c06719e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:09.107059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28953001-0269-474b-a1c1-d1c06719e3f5 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:09.108339 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 694/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:09.113956 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539256-7f2b-40b2-abf4-8ef12938e0d9 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:09.114431 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539256-7f2b-40b2-abf4-8ef12938e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:09.114532 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf539256-7f2b-40b2-abf4-8ef12938e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:09.134582 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-cf539256-7f2b-40b2-abf4-8ef12938e0d9 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:09.135796 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-cf539256-7f2b-40b2-abf4-8ef12938e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:09.144306 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 668/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:09.189811 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28953001-0269-474b-a1c1-d1c06719e3f5 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:09.216720 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28953001-0269-474b-a1c1-d1c06719e3f5 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:09.222526 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdccfb4336b7491d8797c3f682ea0687', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:09.223190 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:09.223480 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:09.223764 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:09.233048 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:09.234686 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa5781-6c2f-4aa8-ab27-022dc26a240c service nova] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:09.245651 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 695/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:17:11.888644 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82d9b-4d38-479a-b15e-c21ebe397572 None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.889231 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82d9b-4d38-479a-b15e-c21ebe397572 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.889420 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82d9b-4d38-479a-b15e-c21ebe397572 None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.890989 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 669/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:11.897204 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10014435-fccc-4335-9b6f-7efa82f0de09 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.897629 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10014435-fccc-4335-9b6f-7efa82f0de09 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.897712 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10014435-fccc-4335-9b6f-7efa82f0de09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.918670 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-10014435-fccc-4335-9b6f-7efa82f0de09 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:11.919908 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-10014435-fccc-4335-9b6f-7efa82f0de09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:11.929360 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 696/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:11.977242 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e174c0-7bf2-4a56-998d-86b5f1c6b6cc None None] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.977536 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e174c0-7bf2-4a56-998d-86b5f1c6b6cc None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.978019 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e174c0-7bf2-4a56-998d-86b5f1c6b6cc None None] PATH_INFO: `/` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:11.979548 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 670/1366] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:11.985059 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dd72f-cdcb-4090-95d1-5b69649f3bf9 None None] REQUEST_METHOD: `POST` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:11.985357 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dd72f-cdcb-4090-95d1-5b69649f3bf9 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:11.985646 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999dd72f-cdcb-4090-95d1-5b69649f3bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:12.008427 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.auth.core [None req-999dd72f-cdcb-4090-95d1-5b69649f3bf9 None None] MFA Rules not processed for user `bdccfb4336b7491d8797c3f682ea0687`. Rule list: `[]` (Enabled: `True`). {{(pid=214148) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:12.009893 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.fernet_utils [None req-999dd72f-cdcb-4090-95d1-5b69649f3bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214148) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:12.020117 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 697/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:17:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:12.028307 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e174c0-7bf2-4a56-998d-86b5f1c6b6cc None None] Authenticating user token {{(pid=214149) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:12.043824 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e174c0-7bf2-4a56-998d-86b5f1c6b6cc None None] Validating token access rules against request {{(pid=214149) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:12.049398 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdccfb4336b7491d8797c3f682ea0687', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214149) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:12.050055 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] REQUEST_METHOD: `GET` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:12.050344 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:12.050616 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:12.061605 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:12.063288 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1f984-a0ee-445d-af7a-c976960ea89c service nova] RBAC: Authorization granted {{(pid=214149) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:12.075983 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 671/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:17:13.145213 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ed658-fdc2-4f4a-ae8f-98784d5da08e None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:13.145819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ed658-fdc2-4f4a-ae8f-98784d5da08e None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:13.145819 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ed658-fdc2-4f4a-ae8f-98784d5da08e None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:13.147783 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 698/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:17:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:13.153354 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8c05-2082-4875-927b-e834ab3fc1a1 None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:13.153972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8c05-2082-4875-927b-e834ab3fc1a1 None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:13.153972 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d8c05-2082-4875-927b-e834ab3fc1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:13.173680 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-e99d8c05-2082-4875-927b-e834ab3fc1a1 None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:13.174890 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-e99d8c05-2082-4875-927b-e834ab3fc1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:13.185382 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 672/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:14.429995 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bb56e-3bca-4b47-95fa-ed4d7ba5c353 None None] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.430553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bb56e-3bca-4b47-95fa-ed4d7ba5c353 None None] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.430553 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712bb56e-3bca-4b47-95fa-ed4d7ba5c353 None None] PATH_INFO: `/` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.431986 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 699/1371] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:17:14.437795 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae48d-02e8-44ec-bea3-704b7a0a2ebc None None] REQUEST_METHOD: `POST` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.438245 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae48d-02e8-44ec-bea3-704b7a0a2ebc None None] SCRIPT_NAME: `/identity` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.438350 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ae48d-02e8-44ec-bea3-704b7a0a2ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=214149) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.457101 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.auth.core [None req-6f7ae48d-02e8-44ec-bea3-704b7a0a2ebc None None] MFA Rules not processed for user `bfca2eff03f04a12a3f7eb5950753314`. Rule list: `[]` (Enabled: `True`). {{(pid=214149) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:17:14.458236 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG keystone.common.fernet_utils [None req-6f7ae48d-02e8-44ec-bea3-704b7a0a2ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=214149) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:17:14.467547 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: [pid: 214149|app: 0|req: 673/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:17:14.519419 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bb56e-3bca-4b47-95fa-ed4d7ba5c353 None None] Authenticating user token {{(pid=214148) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:17:14.535494 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712bb56e-3bca-4b47-95fa-ed4d7ba5c353 None None] Validating token access rules against request {{(pid=214148) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:17:14.542046 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdccfb4336b7491d8797c3f682ea0687', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5b0478445f54f2286808b9731547668', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=214148) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:17:14.542711 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] REQUEST_METHOD: `GET` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:17:14.542996 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] SCRIPT_NAME: `/identity` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:17:14.543281 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=214148) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:17:14.552848 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:17:14.554440 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b990f-45e3-4fae-8d0d-a25de8aad521 service nova] RBAC: Authorization granted {{(pid=214148) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:17:14.565616 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: [pid: 214148|app: 0|req: 700/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:17:24.451325 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: DEBUG dbcounter [-] [214149] Writing DB stats keystone:SELECT=21 {{(pid=214149) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:24.560822 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: DEBUG dbcounter [-] [214148] Writing DB stats keystone:SELECT=86 {{(pid=214148) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:17:36.313247 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: Thu May 19 20:17:36 2022 - graceful shutdown triggered... May 19 20:17:36.313732 dv-cinder-840678-11-iscsi devstack@keystone.service[214148]: Gracefully killing worker 1 (pid: 214148)... May 19 20:17:36.313376 dv-cinder-840678-11-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:17:36.314241 dv-cinder-840678-11-iscsi devstack@keystone.service[214149]: Gracefully killing worker 2 (pid: 214149)... May 19 20:17:37.314205 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: worker 1 buried after 1 seconds May 19 20:17:37.314205 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: worker 2 buried after 1 seconds May 19 20:17:37.314205 dv-cinder-840678-11-iscsi devstack@keystone.service[214136]: goodbye to uWSGI. May 19 20:17:37.315971 dv-cinder-840678-11-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:17:37.316559 dv-cinder-840678-11-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:23:41.149228 dv-cinder-840678-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:23:41.158747 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:23:41 2022] *** May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: nodename: dv-cinder-840678-11-iscsi May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: machine: x86_64 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: clock source: unix May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: pcre jit disabled May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: detected number of CPU cores: 4 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: current working directory: / May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: detected binary path: /usr/bin/uwsgi-core May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: your processes number limit is 47680 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: your memory page size is 4096 bytes May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: detected max file descriptor number: 2048 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: lock engine: pthread robust mutexes May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: thunder lock: enabled May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:23:41.161275 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:23:41.206211 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: Python main interpreter initialized at 0x561c891a09c0 May 19 20:23:41.206211 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: python threads support enabled May 19 20:23:41.206211 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: your server socket listen backlog is limited to 100 connections May 19 20:23:41.206211 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: your mercy for graceful operations on workers is 90 seconds May 19 20:23:41.206697 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:23:41.206697 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: *** Operational MODE: preforking *** May 19 20:23:41.206917 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: *** uWSGI is running in multiple interpreter mode *** May 19 20:23:41.206917 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: spawned uWSGI master process (pid: 277564) May 19 20:23:41.207132 dv-cinder-840678-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:23:41.208175 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: spawned uWSGI worker 1 (pid: 277574, cores: 1) May 19 20:23:41.208175 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: spawned uWSGI worker 2 (pid: 277575, cores: 1) May 19 20:23:41.209673 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:23:47.005778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:23:47.005778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: from cryptography.utils import int_from_bytes May 19 20:23:47.006718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:23:47.006718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: from cryptography.utils import int_from_bytes May 19 20:23:47.031119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:23:47.031119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: from cryptography.utils import int_from_bytes May 19 20:23:47.031928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:23:47.031928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: from cryptography.utils import int_from_bytes May 19 20:23:47.057287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:23:47.057477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:23:47.057653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] command line args: [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:23:47.057928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:23:47.058103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ================================================================================ {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:23:47.058358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] admin_token = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.058604 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] config_dir = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.058791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.058974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] config_source = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.059154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] debug = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.059352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.059708 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.059892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.060146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.060429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.060648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.060830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_config_append = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.061010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.061189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_dir = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.061368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_file = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.061607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.061859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.062129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.062357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.062555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.062790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.063070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.063783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.064224 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.064723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.065195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.065648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.066128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.066474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.066771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.067277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.067468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] publish_errors = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.067709 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.067960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.068103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.068359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.068628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.068871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] standard_threads = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.069106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.069355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.069534 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.069712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] use_journal = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.069889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] use_json = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.070066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] use_stderr = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.070243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] use_syslog = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.070477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.070738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.070981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.071246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.071488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.071743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.071984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.072236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.072497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.072750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.073006 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.073261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.073496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.073745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.073981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.074229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.074476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.074717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.075065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.075331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.075585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.075831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.076108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.076376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.076658 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.076913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.077148 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.077390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.077624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.077864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.078127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.078366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.079121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.079357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.079610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.079842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.080090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.080343 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.080606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.080849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.080992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:23:47.081144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.081291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:23:47.081436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.081577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] command line args: [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:23:47.081710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.081856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:23:47.082010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.082163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ================================================================================ {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:23:47.082297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.082436 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] admin_token = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.082576 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.082715 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] config_dir = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.082853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.083008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.083008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] config_source = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.083274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.083415 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] debug = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.083555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.083710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.083878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.084018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.084157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.084298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] insecure_debug = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.084474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.084631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.084773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.084918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.085058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.085196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.085330 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.085474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_config_append = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.085613 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.085752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.085752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_dir = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.086014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.086160 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_file = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.086300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.086439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.086594 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.086740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.086880 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.087034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.087175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.087313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.087450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.087588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.087728 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.087869 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.088004 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.088143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.088295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.088472 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.088621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.088760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.088900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.089038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.089200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.089341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.089482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.089621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.089753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.089891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.090036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.090175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] notification_format = cadf {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.090314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.090452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.090593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.090593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.090865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.091004 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.091143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] publish_errors = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.091289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.091437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.091577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.091710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.091710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.091976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.091976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.092238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.092238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.092524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.092665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] standard_threads = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.092804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.092804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.093117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] strict_password_check = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.093257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.093396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.093396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] use_eventlog = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.093661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.093799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] use_journal = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.093978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.094125 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] use_json = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.094125 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] use_stderr = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.095086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.095234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] use_syslog = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.095384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.095536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] watch_log_file = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:23:47.095677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.095829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.095976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.096888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097449 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.097853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098004 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.098988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.099991 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.external = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.100965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.101974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.password = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] auth.token = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.102983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103548 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.103945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.104954 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105379 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.105940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.106925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107132 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107708 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.107979 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.108890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.109920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.110979 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.111955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.112971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.113967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.114794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115334 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115334 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.115909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116176 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.116914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117054 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117612 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.enabled = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.117891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118296 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118296 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118696 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118834 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118966 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.118966 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.119909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.120904 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.121769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.122841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.123972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.124936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.125091 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.125226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.125358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.126851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.127037 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.127276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.127577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.127855 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.128129 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.external = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.128370 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.128579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.mapped = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.128767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.128959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.129146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.129330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.129506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.129688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.password = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.129870 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.130068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] auth.token = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.130273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.130457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.130633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.130818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] catalog.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131749 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.131927 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.132111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.132304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.132595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.132780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.133031 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.133289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.133549 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.133827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.134081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.134282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.134476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.134667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.134863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.135068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.135261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.135449 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.135643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.135830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.136790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.137158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.137158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.137528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.137719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.137901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.138774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139615 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.139816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.140059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.140295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.140570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.140779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.141013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.141853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.142090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.142313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.142527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.142744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.142953 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.143174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.143399 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.143629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.143849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.144080 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.144301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.144550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.144779 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.145011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.145241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.145461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.145696 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] role.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.145917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.146151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] role.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.146359 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.146582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.146803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.147021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.147241 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.147470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.147694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.148059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.148289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.148579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.148821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.149054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.149266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.149510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.149723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.149917 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.150117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.150332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.150535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.150750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.150904 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151046 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.151893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.152896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153728 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.153861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154140 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154546 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.154954 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.155914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.156047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.156200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.156341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.156560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.156781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157530 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.password = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157862 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.157995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158417 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.158961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159537 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.159826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.160908 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161048 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161728 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.161873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162299 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162299 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.162986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.connection = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.163984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164131 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.164996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.165961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166247 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166683 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.166981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167265 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.167981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168262 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168440 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] policy.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.168588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] receipt.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.169752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] revoke.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] role.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] role.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] role.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.170887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] role.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.172041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.173293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.174454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.provider = fernet {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.175580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] trust.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.176762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.connection = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.177895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.179146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:23:47.180290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=277575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:23:47.181218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.181218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.181218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.191658 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.197864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.198034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.198192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.209462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.222938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.223177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.228068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.228309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.277580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.278737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.330149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.330397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.330962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.331207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.398429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.398698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.398803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.398895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.398983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399172 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.399635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.423948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424418 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424860 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.424965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.425082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.425185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.425284 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.425380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.425501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.436823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.437154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.450425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.450681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.450878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.451108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.451334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.451668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.451978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.452271 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.452541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.452769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.452991 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.453208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.453424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.454628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=277574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:23:47.469986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.470229 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:23:47.470422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.470525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:23:47.470668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.470839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:23:47.470943 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:23:47.471137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471407 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.471932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.472082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.484043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.484319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.484615 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.485537 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.485656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.485825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.486096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.486475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.493991 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.497059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.497425 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.497721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.498009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.498430 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.498721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.498988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.499263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.499544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.499819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.500094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.500366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.501801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.502100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.502372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.502786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.503075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.503342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.503616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.503894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.504171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.504478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.504756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.527718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.529379 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.529661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.532674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.534303 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.534588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.535644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.535924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.536203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.539908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.540187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.540496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.543641 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.546628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.548969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.549388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.549891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.550196 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.550480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.550760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.551036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.551312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.551585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.551858 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.555238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.555535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.555825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.556111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.556388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.556694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.556969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.557244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.571501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.571782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.576327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.577111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.577397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.580078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.580367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.582159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.585348 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.585816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.586116 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.588946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.591105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.592821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.593153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.593473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.593793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.594099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.594782 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.594999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.598671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.598989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.599285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.599572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.599858 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.600145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.614961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.615362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.615635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.615907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.620216 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.620645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.620923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.621204 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.623877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.626231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.626511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.629592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.629875 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.629979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.630162 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.630446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.630724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.631610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.631892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.634976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.635261 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.635555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.635858 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.636143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.644384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.644695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.644976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.645261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.645536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.645810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.646097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.650094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.650282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.650591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.650879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.651165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.651446 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.651732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.664666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.667966 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.670301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.671337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.673530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.674163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.675268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.675554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.675841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.677619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.679234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.680986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.681343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.681511 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.683310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.684970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.689129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.689475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.689762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.689858 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.690039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.691512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.694812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.695107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.695400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.695684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.699616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.699922 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.700211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.705426 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.705730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.706020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.708618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.708914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.709197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.709483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.709763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.710051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.710327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.710597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.710867 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.711153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.711424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.711684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.711954 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.712220 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.714175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=277575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:23:47.714466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.714748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.715025 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.715307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.715586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.715870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.716147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.716437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.716716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.717019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.717293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.717562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.717823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=277575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:23:47.752308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c891a09c0 pid: 277574 (default app) May 19 20:23:47.756039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563c5d-3d83-4df6-93c1-f727c5fbfb12 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:47.756341 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563c5d-3d83-4df6-93c1-f727c5fbfb12 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:47.756650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe563c5d-3d83-4df6-93c1-f727c5fbfb12 None None] PATH_INFO: `/v3` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:47.757518 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c891a09c0 pid: 277575 (default app) May 19 20:23:47.761018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu May 19 20:23:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:23:55.992335 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:55.992915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:55.992915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:55.997466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:23:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:23:56.003592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.004090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.004199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.074981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: INFO dbcounter [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] Registered counter for database keystone May 19 20:23:56.106882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG oslo_db.sqlalchemy.engines [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277574) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:23:56.113497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writer thread running {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:23:56.316626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG passlib.handlers.bcrypt [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277574) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:23:56.317067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG passlib.handlers.bcrypt [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277574) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:23:56.374423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:56.376245 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:56.518053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:23:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:23:56.541453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:56.612262 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: INFO dbcounter [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] Registered counter for database keystone May 19 20:23:56.633860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG oslo_db.sqlalchemy.engines [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=277575) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:23:56.640270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writer thread running {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:23:56.765132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed386ec-ef9d-4c05-90b6-2a124dfdb99e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:56.770286 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4631908c3a341389e87c50c9ca6a7c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:56.771007 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.771305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.771598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.780941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:56.833624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:23:56.833963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f982ea-22c3-4879-8109-9a14204c511e admin admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:56.844832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 20:23:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:23:56.851278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:56.869161 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd6156-a6b8-4d78-8ea9-2c790feab8d8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:56.873825 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4631908c3a341389e87c50c9ca6a7c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:56.874984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:56.875904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:56.876225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] PATH_INFO: `/v3/projects/b4631908c3a341389e87c50c9ca6a7c4` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:56.878005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] RBAC: Authorizing `identity:get_project(project_id=b4631908c3a341389e87c50c9ca6a7c4)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:57.040871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: WARNING keystone.common.rbac_enforcer.enforcer [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:23:57.041588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf08a9-f8a8-44ef-8fbd-69552a1294fd admin admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:57.046076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu May 19 20:23:56 2022] GET /identity/v3/projects/b4631908c3a341389e87c50c9ca6a7c4 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:23:59.385779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7374a-5dfc-4066-81e6-d60a511c7130 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:59.386344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7374a-5dfc-4066-81e6-d60a511c7130 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:59.386344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7374a-5dfc-4066-81e6-d60a511c7130 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:59.390568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:23:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:23:59.396917 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:59.397398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:59.397514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:59.417223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:23:59.418744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:23:59.427509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:23:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:23:59.451381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7374a-5dfc-4066-81e6-d60a511c7130 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:59.464590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7374a-5dfc-4066-81e6-d60a511c7130 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:59.468992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4631908c3a341389e87c50c9ca6a7c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:59.469774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:59.470064 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:59.470347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:59.478165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:59.479962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d25e6-ac3f-48c2-9a10-b178624cb459 admin admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:59.489503 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu May 19 20:23:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:23:59.495526 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:23:59.505388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fa040-9caf-4adf-a6d2-7f9b863a91e5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:23:59.509818 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b4631908c3a341389e87c50c9ca6a7c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:23:59.510684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:23:59.510975 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:23:59.511258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:23:59.512695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:23:59.514531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fa733-1c18-4c3e-899e-201287229676 admin admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:23:59.516847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu May 19 20:23:59 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:01.832200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8428f-6dbb-4a36-ad44-9f2f4c117a4f None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:01.832857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8428f-6dbb-4a36-ad44-9f2f4c117a4f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:01.832857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8428f-6dbb-4a36-ad44-9f2f4c117a4f None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:01.834226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:01.840078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:01.840598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:01.840716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:01.859064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:01.860325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:01.901948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee9f8-3a2d-4fb3-98ac-4235d591e636 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:01.902233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee9f8-3a2d-4fb3-98ac-4235d591e636 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:01.902520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dee9f8-3a2d-4fb3-98ac-4235d591e636 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:01.903908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:01.910055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:01.910343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:01.910627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:01.951564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG passlib.handlers.bcrypt [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=277575) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:24:01.951986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG passlib.handlers.bcrypt [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=277575) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:24:01.955220 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:01.978237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.001238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:02.002890 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:02.004619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.089338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38ab23-46c2-476b-9d91-666b273aca2e None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.108178 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:02.129715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.130544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.130838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.131127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.131315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.144088 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:02.155734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:02.157941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:02.158965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:02.163401 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.253106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.257192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13f938-bbf0-4509-9c0c-97ac50c9e866 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.259906 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.277513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cfdd0-6b85-47ee-bf6b-9b884407bc90 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.299581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.300560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.300855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.301139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.318790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.319611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.319902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.320197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.321012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:02.321334 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.322752 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.335832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:02.341275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.373128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.374647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.454746 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64509269-fd89-43b9-9901-3a56b0cd40b1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.480434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:24:02.485853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.488709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.494546 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.495523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.495886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.496263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.497619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.499112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.501483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:02.506628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.522722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.601855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adaf37-ab3f-4a32-a0f6-8623f4f2aa81 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.635507 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca5084-4035-4af4-ae10-4801ea631689 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.639502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.640344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.640661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.641063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.660462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.661983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.673011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.673965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.674263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.674552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.675929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.677486 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.679203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:02.684971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:24:02.687667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:02.689779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:02.691356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:24:02.691356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:24:02.691356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:24:02.691356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:02.691356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:02.696205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:02.756667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.775606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:02.797616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ac2b8-75c3-4c07-9a14-710d6c990452 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.811523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4658d35-e040-4141-8dbd-f1a24ec36ec6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:02.833452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.834130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.834405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.834675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.835529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.domain.id=default, user.id=a86a18b340304607b0aafdfa7ad74e79) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:02.835842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.837387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bc98a-a400-4a53-86bc-c1c3429ec111 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.844513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/role_assignments?user.id=a86a18b340304607b0aafdfa7ad74e79&scope.domain.id=default&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:02.850682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:02.851471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:02.851803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:02.852147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:02.852907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:02.853225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:02.854808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3e30-8bd9-4d9c-a7d0-7a993fcb4214 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:02.878491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:05.480016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6c7f4-f684-473e-8649-85dabdee10ec None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:05.481024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6c7f4-f684-473e-8649-85dabdee10ec None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:05.481472 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6c7f4-f684-473e-8649-85dabdee10ec None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:05.483138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:05.488988 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0970b-25b2-42fe-b1a1-67e722703ed5 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:05.489806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0970b-25b2-42fe-b1a1-67e722703ed5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:05.490283 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0970b-25b2-42fe-b1a1-67e722703ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:05.633666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-abc0970b-25b2-42fe-b1a1-67e722703ed5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:05.635177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-abc0970b-25b2-42fe-b1a1-67e722703ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:05.726882 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 11/23] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:05.774696 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6c7f4-f684-473e-8649-85dabdee10ec None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:05.801099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a68da2-2789-4046-97aa-b45dbc2609aa None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:05.801640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a68da2-2789-4046-97aa-b45dbc2609aa None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:05.802101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a68da2-2789-4046-97aa-b45dbc2609aa None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:05.803654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:05.808886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3c87-cb9d-4aa4-8442-5e9ea2dd33d7 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:05.809413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3c87-cb9d-4aa4-8442-5e9ea2dd33d7 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:05.809878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3c87-cb9d-4aa4-8442-5e9ea2dd33d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:05.841961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-c20c3c87-cb9d-4aa4-8442-5e9ea2dd33d7 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:05.845482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-c20c3c87-cb9d-4aa4-8442-5e9ea2dd33d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:05.964808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:06.014412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 20:24:06.016581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af338-4591-47d4-b026-f8db013af142 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.018252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6c7f4-f684-473e-8649-85dabdee10ec None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.020570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af338-4591-47d4-b026-f8db013af142 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.021113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328af338-4591-47d4-b026-f8db013af142 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.028625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 14/26] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:06.030487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.031201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.032504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.064049 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:06.065709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:06.100639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.105038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.105673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.106167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.129163 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:06.137638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:06.138639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:06.139590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:06.142381 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 13/27] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:06.143438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.153195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:06.244719 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 15/28] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:06.246791 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.352084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0c8a5-3ed4-4691-8bfe-e355181aa141 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.373836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd520e2-13ad-4807-bbbc-1e6cf7d955ee None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.388708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:06.405643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.407633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.408140 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.408654 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.422759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:06.424883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:06.436371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.437247 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.437589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.438666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.439461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:06.439788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:06.441263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af397ba6-c7ba-4d91-86ea-09881cec8deb None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:24:06.448456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:06.449464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:06.450427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:24:06.450427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:24:06.450427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:24:06.450427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:06.452560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 14/29] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:06.454195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.454954 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 16/30] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:06.460112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c35c4-7ebd-4090-bc69-308898a2c299 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.460112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c35c4-7ebd-4090-bc69-308898a2c299 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.460112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c35c4-7ebd-4090-bc69-308898a2c299 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.460112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 17/31] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:06.464202 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c35c4-7ebd-4090-bc69-308898a2c299 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.541415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:06.591593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c35c4-7ebd-4090-bc69-308898a2c299 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.607095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd238e9-03fc-430f-b75b-a9344a86bca7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.635948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.637033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.637559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.638072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.639139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:06.639664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:06.641522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:06.655952 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:06.657096 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.657810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.658107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.658393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.658935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.659176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:06.660652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab31587-ea2b-4831-929b-4bc3f400c91d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:06.699253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:06.794744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076a0a1-d6bf-41c4-802b-4a2aedb49e0f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:06.834190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:06.835608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.836148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.836781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.838778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:06.844461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 15/33] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:24:06 2022] POST /identity/v3/roles => generated 219 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:06.844461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:06.844461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:06.844461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:06.845071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:06.848234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 19/34] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:06 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:06.853141 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:06.880156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:06.881860 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:06.971538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 16/35] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:06.980429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.051480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.053036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.060454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b235-4a26-44ea-bd3a-8f9cdab0230d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.113553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.114621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.115181 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.115684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.116698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.118458 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.139962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986077b-549e-4537-bdc6-70745a7ea738 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.191152 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.192373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.193034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.193595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.195201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.197281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a55aa7-f1b1-46fc-8c63-564edb17531a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.199799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 17/36] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:07.201602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214b96-fd58-43fa-be1e-707f2dae459b None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.202350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214b96-fd58-43fa-be1e-707f2dae459b None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.202896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de214b96-fd58-43fa-be1e-707f2dae459b None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.205624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 18/37] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:07.206677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de214b96-fd58-43fa-be1e-707f2dae459b None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.280952 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 20/38] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:24:06 2022] POST /identity/v3/projects => generated 301 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:07.282575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.337431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de214b96-fd58-43fa-be1e-707f2dae459b None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.356136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.382199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.382906 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.383203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.383498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.384289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.385782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c702d83-a325-444a-84b9-7faa94409299 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.400901 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 19/39] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/roles => generated 225 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:07.402215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e82db-3e77-407f-9c15-4e61994da3d2 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.402518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e82db-3e77-407f-9c15-4e61994da3d2 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.402807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e82db-3e77-407f-9c15-4e61994da3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.423992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-848e82db-3e77-407f-9c15-4e61994da3d2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:07.424674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302abad-cf67-43ba-8ad1-8c033141b776 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.425343 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-848e82db-3e77-407f-9c15-4e61994da3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:07.479117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.480477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.481054 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.481578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] PATH_INFO: `/v3/domains/default/users/a86a18b340304607b0aafdfa7ad74e79/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.485088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a86a18b340304607b0aafdfa7ad74e79, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.487271 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078dfee5-d690-4f41-90d2-6bc8dffa8424 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.515883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 21/40] 10.222.0.13 () {58 vars in 1361 bytes} [Thu May 19 20:24:07 2022] PUT /identity/v3/domains/default/users/a86a18b340304607b0aafdfa7ad74e79/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:07.518073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6651f-f704-445e-bdfa-7bb11206729d None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.518636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6651f-f704-445e-bdfa-7bb11206729d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.520496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 20/41] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:07.525144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6651f-f704-445e-bdfa-7bb11206729d None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.531892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 22/42] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:07.534423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.535077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.535648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.548690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6651f-f704-445e-bdfa-7bb11206729d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.557263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:07.559327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:07.565822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.653743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 21/43] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:07.679452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.712242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6651f-f704-445e-bdfa-7bb11206729d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.729935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:07.765022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.766327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.766662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.766936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.767927 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.769572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.792080 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 23/44] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/roles => generated 223 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:07.821196 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b416b94-8a8f-4973-933c-8483cddf0209 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:07.870155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:07.870992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:07.871443 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:07.872111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:07.874424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:07.876843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfb58-c8bd-41ed-9581-c9c5c4f32134 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:07.880480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 22/45] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:07.886244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:07.965520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:08.020686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc6ebc-a10c-46b1-9d39-43338bd059e4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:08.063229 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:08.064173 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:08.064497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:08.064798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:08.065574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:08.067099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d629771-51e7-44e3-85f3-0a0c722f1ee3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:08.096340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 24/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:24:07 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:09.886837 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7f79-57b9-49b8-a6c9-9b73290842b0 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:09.887399 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7f79-57b9-49b8-a6c9-9b73290842b0 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:09.887725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d7f79-57b9-49b8-a6c9-9b73290842b0 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:09.889252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 23/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:09.895369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:09.895976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:09.895976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:09.913968 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:09.915539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:09.987456 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 25/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:10.010055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d7f79-57b9-49b8-a6c9-9b73290842b0 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.118726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d7f79-57b9-49b8-a6c9-9b73290842b0 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.120458 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.157319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.158136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.158428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.158715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.169082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:10.177377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:10.178513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:10.179447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 24/49] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:10.184699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.238075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.295898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf1bdd-3526-4618-8469-5e66f3927e3f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.331709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.332459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.332759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.333047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.333846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:10.334166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.335622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb19236-b9ba-48fe-a272-3a37c203bc08 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.345183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 26/50] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:10.351075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.438227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8096cc-74a8-4682-bc22-0505f97dc7b9 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.438809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8096cc-74a8-4682-bc22-0505f97dc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.439105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8096cc-74a8-4682-bc22-0505f97dc7b9 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.440574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 27/51] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:10.446167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.446466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.446758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.459205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e7e1c-aee0-4d1e-8b42-427ef7073282 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.463436 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:10.464679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:10.495315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.496213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.496533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.496820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.498049 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.499521 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.501657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 25/52] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:10.507011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.534346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 28/53] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:10.539543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.558523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.606427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.616390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b088-2928-4e26-ac8b-764edc93bf3c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.663815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.664668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.664976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.665263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.666514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.667729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee5f6-bb0b-408a-82f3-d511ce33ee5a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.667992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.669709 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 26/54] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:10.676056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.715885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.716637 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.716935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.717230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.718457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.720473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.722148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:10.727726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:10.784016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.787874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b95ce9-7705-4924-9730-4f63e0190433 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.824525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.825201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.825499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.825800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.826648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.domain.id=default, user.id=a86a18b340304607b0aafdfa7ad74e79) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:10.826974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.829031 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e993b3a-45d9-4c85-ab7a-3d82b2bf9064 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.835047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 27/56] 10.222.0.13 () {56 vars in 1322 bytes} [Thu May 19 20:24:10 2022] GET /identity/v3/role_assignments?user.id=a86a18b340304607b0aafdfa7ad74e79&scope.domain.id=default&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:10.845744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1998a05-54f9-45ed-b462-ed5213577979 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:10.860056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:10.886910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:10.887580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:10.887876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:10.888165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:10.888961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:10.890467 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e6ad4-b291-4428-ac0f-b4060c2cb63a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:10.921414 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 30/57] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 20:24:10 2022] POST /identity/v3/projects => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:13.211401 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b28f9-9882-43c2-a282-d9b2bf33ab40 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:13.211978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b28f9-9882-43c2-a282-d9b2bf33ab40 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:13.211978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b28f9-9882-43c2-a282-d9b2bf33ab40 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:13.213401 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:13.218665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:13.219254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:13.219254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:13.236555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:13.237780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:13.310068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:13.335392 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b28f9-9882-43c2-a282-d9b2bf33ab40 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:13.445962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b28f9-9882-43c2-a282-d9b2bf33ab40 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:13.492795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:13.493531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:13.493825 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:13.494111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:13.495335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:13.496867 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0f46a-052b-4529-ab8b-12a8ab1b46e8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:13.498531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:13.504887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:13.608289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:13.619286 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bd82c-bb04-4cc4-96bc-67bcb9d2b41b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:13.658137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:13.659291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:13.659291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:13.659516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:13.660099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:13.661977 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01e004-1b46-49e6-8b01-48019f36bf56 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:13.689807 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:13 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:17.170802 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da880-3279-4f31-b6d2-14f4352ca2b5 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.171392 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da880-3279-4f31-b6d2-14f4352ca2b5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.171392 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5da880-3279-4f31-b6d2-14f4352ca2b5 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.173223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:17.179255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f5b7-e991-4464-9410-0fb83734054d None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.179782 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f5b7-e991-4464-9410-0fb83734054d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.179864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525f5b7-e991-4464-9410-0fb83734054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.205702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-9525f5b7-e991-4464-9410-0fb83734054d None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:17.207423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-9525f5b7-e991-4464-9410-0fb83734054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:17.310819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 33/63] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:17.363290 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5da880-3279-4f31-b6d2-14f4352ca2b5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:17.374390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902d1ad-6126-420e-b4f3-1953c3bc1500 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.375666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902d1ad-6126-420e-b4f3-1953c3bc1500 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.376304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902d1ad-6126-420e-b4f3-1953c3bc1500 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.378151 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 34/64] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:17.384090 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.384386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.384907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.409085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:17.410295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:17.420780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:17.567114 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 35/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:17.590616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:17.593380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5da880-3279-4f31-b6d2-14f4352ca2b5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:17.616787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:17.652645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:17.653482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.653775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.654064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.678197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-c7ad9af1-6461-44a4-a2b4-d1b5e053a39e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:17.698726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:17.699961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 31/66] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:17.701864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabd38-4a67-4419-a503-3fadf502dddf None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.702932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabd38-4a67-4419-a503-3fadf502dddf None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.702932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabd38-4a67-4419-a503-3fadf502dddf None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.704150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 32/67] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:17.713849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabd38-4a67-4419-a503-3fadf502dddf None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:17.721313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c3a30-03c7-4812-b9b4-ed1248abfd71 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:17.764689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:17.766148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.766662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.767155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.779760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2165084d-0512-42a4-a48a-249b1aac136c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:17.789913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:17.791235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:17.792350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:17.794732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 36/68] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:17.796274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6bb0-a5d8-4b9f-9775-6f0fa0da3f91 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.796772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6bb0-a5d8-4b9f-9775-6f0fa0da3f91 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.797205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b6bb0-a5d8-4b9f-9775-6f0fa0da3f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.816424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a36b6bb0-a5d8-4b9f-9775-6f0fa0da3f91 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:17.817734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a36b6bb0-a5d8-4b9f-9775-6f0fa0da3f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:17.848951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:17.865752 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabd38-4a67-4419-a503-3fadf502dddf None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:17.892498 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:17.899835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 37/69] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:17.901289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216de2-cfc4-405c-829e-bc6c0f3c76f1 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.901590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216de2-cfc4-405c-829e-bc6c0f3c76f1 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.901881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216de2-cfc4-405c-829e-bc6c0f3c76f1 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.903186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 38/70] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:17.908763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216de2-cfc4-405c-829e-bc6c0f3c76f1 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:17.915183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:17.916001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.916488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.916810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.917712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:17.918136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:17.920223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5229c7-273f-44ed-829a-9cdeb1bad3ac None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:17.932473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 33/71] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:17.933792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:17.933926 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:17.934183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:17.955494 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:17.956771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:18.024999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216de2-cfc4-405c-829e-bc6c0f3c76f1 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.040464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 34/72] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:18.040464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.078886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.079496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.079794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.080087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.080901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:18.081227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.082662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c2d62-4ddc-4639-b417-60e633dacc1d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.091221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 39/73] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:18.097108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e984-02a4-4fd8-8e14-ae51c8040b55 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.097108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e984-02a4-4fd8-8e14-ae51c8040b55 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.097108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48e984-02a4-4fd8-8e14-ae51c8040b55 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.097108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 40/74] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:18.097108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e984-02a4-4fd8-8e14-ae51c8040b55 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.134064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.153731 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b2756-774e-4052-bdf5-90ab9e0ae821 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.164473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.191528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.192366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.192690 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.192981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.204707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:18.209987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48e984-02a4-4fd8-8e14-ae51c8040b55 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:18.213169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:18.214205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 35/75] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:18.220964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.249166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.250652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.250980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.251265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.268340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.269847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.271774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 41/76] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:18.272381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.333832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a371c-5aff-42a6-b242-8304a88bb995 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.369947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.370773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.371072 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.371350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.383756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:18.389845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32212d9d-9330-4ce9-873d-c9732637d9cc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5f291d29-eef1-4643-8e11-c4d63f16fa6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:18.392475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:18.393610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:18.394665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 36/77] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:18.395514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.395514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.395661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.403045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.414653 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.416971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:18.418236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:18.434087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.434976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.435264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.435549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.436785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.438624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.440298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 42/78] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:18.441099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.492658 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 37/79] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:18.493695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.561704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda350-8792-4638-b633-f546f5cecf9f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.598629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.599223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.599520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.599814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.600606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:18.600931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.602326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.609445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d047bf5-2f7c-45d0-96d1-cba5992fa1b9 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.610702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 43/80] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:18.615563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.647367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.653565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.654252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.654551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.654842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.667932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.732218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ed5a4-4f4e-4ac6-bf5a-b03857770df9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.733666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.736587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.769528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.770502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.770794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.771079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.771835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:18.772151 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:18.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:18.783528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 44/81] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:18.789431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.852843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 38/82] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:18.853716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:18.890361 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.899970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547b89-d1a1-4cf5-866c-f51dad62ac80 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:18.910864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:18.935788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:18.936522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:18.936824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:18.937117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:18.965135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1ab33-ac39-49dd-80af-3214697561b7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.001715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.002529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.002972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.003123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.004295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.005795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.007450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 39/83] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:19.008222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.012092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.013648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.113367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.116575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c66ad-31fd-4d8a-85b0-e2e7112f5bea None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.119228 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 45/84] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:19.119941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.132113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.160367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.161282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.161569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.161848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.163084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.165016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.167078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 40/85] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:19.168171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.229283 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3c3f5-f0c5-4f20-9464-7ae84cd1a422 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.267310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.268194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.268520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.268806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.270042 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.271515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.273231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 46/86] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:19.274074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.289371 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc537c-5fe0-4c29-9370-0084fa913c4e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.325750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.326631 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.326931 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.327215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.329010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.330508 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.332252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 41/87] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:19.333059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.368334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.381265 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.392185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ae488-cf7b-4aa2-81a2-9e82358f78fe None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.433588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.434276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.434570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.434857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.435590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.437104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.456009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcfd03-c5b5-41e6-a9ef-f794b91f6405 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.460754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 47/88] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:19 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:19.466345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.499631 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.501225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.501523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.501815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.503147 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.504726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.515503 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 42/89] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:19.515503 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.599411 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73872f72-e13d-4c3f-aae8-db5aa5e930c3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.634704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f531da-6391-4c23-89b7-82353d59f6f1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:19.637266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.639949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.641523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.641824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.642225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.645698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.688878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:19.690657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:19.690657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:19.690657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:19.743247 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.744799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.780699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:19.780699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:19.876820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 48/90] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:19.878173 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.913371 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 43/91] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:19.914554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:19.918756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:19.921465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:20.020298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dca290-8446-477e-b0b2-6970289e570e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.059879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674e4a4-8894-45eb-af73-ea76060da910 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.074259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.075843 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.076376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.076884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.078839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=50716494befb4be495382d33e183829c) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:20.079797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.081930 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.099221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 49/92] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:20.105549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:20.122322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.123029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.123327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.123619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.124606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=a86a18b340304607b0aafdfa7ad74e79) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:20.125611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.127188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.137584 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 44/93] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:19 2022] GET /identity/v3/role_assignments?user.id=a86a18b340304607b0aafdfa7ad74e79&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:20.139948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:20.324524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:20.362313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:20.367860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b714e-0207-44d9-8ff0-950af0076e12 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.422694 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a735c02-5421-454d-8ee9-58bc1899046d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.450933 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02882806-0838-4112-af54-5665fe50053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.451868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02882806-0838-4112-af54-5665fe50053c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.452165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02882806-0838-4112-af54-5665fe50053c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.452482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02882806-0838-4112-af54-5665fe50053c None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.456796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02882806-0838-4112-af54-5665fe50053c None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.458972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02882806-0838-4112-af54-5665fe50053c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.461319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 50/94] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:20 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:20.467386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02882806-0838-4112-af54-5665fe50053c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:20.541190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.542102 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.542398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.542687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] PATH_INFO: `/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.545155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] RBAC: Authorizing `identity:get_project(project_id=e04b1cdfbeb6460bb579d7328e46ce04)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.547183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.550455 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 45/95] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:20 2022] GET /identity/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04 => generated 312 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:20.560563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:20.660290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02882806-0838-4112-af54-5665fe50053c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.754446 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.755138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.755439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.755727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.760751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=50716494befb4be495382d33e183829c) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:20.761139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.762681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d01e98-b303-4e7a-9f66-1cf4894db935 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.777133 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 51/96] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:20 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:20.801753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599d1de-a0d4-4509-b476-dd914b90c15a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:20.863201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:20.863935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:20.864236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:20.864559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:20.865448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=e04b1cdfbeb6460bb579d7328e46ce04, user.id=50716494befb4be495382d33e183829c) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:20.865771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:20.867317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f872d0-e61c-46d9-846d-a0178144555e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:20.874256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:20.878806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 46/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:20 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=e04b1cdfbeb6460bb579d7328e46ce04&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 259 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:22.321652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39cd02-d690-4a39-8748-966684ff7844 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:22.322226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39cd02-d690-4a39-8748-966684ff7844 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:22.322226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39cd02-d690-4a39-8748-966684ff7844 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:22.324033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 52/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:22.330718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:22.331263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:22.331356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:22.361617 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:22.362882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:22.454803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 47/99] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:22.484012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39cd02-d690-4a39-8748-966684ff7844 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:22.526154 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:22.618187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39cd02-d690-4a39-8748-966684ff7844 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:22.662521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:22.663764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:22.664311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:22.664837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:22.680775 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-323352f3-5a71-4f75-ae95-9d7d764635fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:22.694766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:22.696067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:22.701508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:22.704668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 53/100] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:22.710529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:22.897454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782260a7-bed8-4988-8ad3-98bcfb19e020 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:22.897977 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782260a7-bed8-4988-8ad3-98bcfb19e020 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:22.898496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782260a7-bed8-4988-8ad3-98bcfb19e020 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:22.901157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 54/101] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:22.916529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:22.917050 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:22.917562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:22.960814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:22.962891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:22.981236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbb9ec-a579-45aa-9481-e9a75af11a84 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:23.081482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:23.101541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:23.102805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.103402 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.103889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.105724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:23.106290 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:23.108921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:23.148618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 48/102] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:23.152043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:23.210375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 55/103] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:23.290651 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:23.352798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:23.432774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fc842-386f-429b-8669-f1637c5d20b4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:23.494826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:23.495736 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.496036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.496328 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] PATH_INFO: `/v3/users/595036af48cc4f96a5426e8f0b6d9ccf` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.530242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] RBAC: Authorizing `identity:get_user(user_id=595036af48cc4f96a5426e8f0b6d9ccf)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:23.531922 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99cf0c-5800-4b58-8bcd-73fc447d7b69 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:23.535159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 49/104] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:23 2022] GET /identity/v3/users/595036af48cc4f96a5426e8f0b6d9ccf => generated 289 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:23.537012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a0d-f786-46db-ae83-f45a66ccd73b None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.537323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a0d-f786-46db-ae83-f45a66ccd73b None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.537620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e5a0d-f786-46db-ae83-f45a66ccd73b None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.539099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 50/105] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:23.544679 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5a0d-f786-46db-ae83-f45a66ccd73b None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:23.592159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b454422-bc92-4b90-a2c7-4819d5ac3952 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:23.641439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:23.668164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:23.669013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.669317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.669605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] PATH_INFO: `/v3/domains/default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.670893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:23.672809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604534-76a2-4ad5-a6cb-cea63b46578d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:23.674910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 56/106] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:23.676895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.677205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.677500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.700462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:23.703488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:23.778669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e5a0d-f786-46db-ae83-f45a66ccd73b None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:23.805226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 57/107] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:23.806611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:23.848465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:23.867911 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:23.869039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:23.869552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:23.870234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:23.957983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:23.959878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:23.977892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7628e96-e014-45d8-ba8e-ba5b9a024d5c None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.017169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:24.036768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.037771 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.038225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.038667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.039584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:24.041445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd972026-f680-489c-8cee-1db5ea154ce1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:24.077614 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 58/108] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:24:23 2022] POST /identity/v3/projects => generated 302 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:24.079164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 51/109] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:24.080465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353803-a028-4dbe-bcf0-ec889a6a8e14 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.081000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353803-a028-4dbe-bcf0-ec889a6a8e14 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.081517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353803-a028-4dbe-bcf0-ec889a6a8e14 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.083269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 59/110] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:24.086013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353803-a028-4dbe-bcf0-ec889a6a8e14 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.086750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.263524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:24.312021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353803-a028-4dbe-bcf0-ec889a6a8e14 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.361824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.362903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.363062 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.363320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.366929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:24.368487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839a042-c231-4558-9cfd-5e72fd5db1b6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:24.370366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 60/111] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:24.371646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.371944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.372241 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.377756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad330397-bcf5-47ea-bd4a-866b33f4c76e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.403318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:24.403688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:24.438042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.440203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.441323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.442274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.471377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:24.484815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:24.486215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:24.487138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 52/112] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:24.487949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.518154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:24.530350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 61/113] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:24.531998 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.617271 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:24.644842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c01f7-5467-4347-9d5f-614a9a449240 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.659303 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4808a72-3343-4664-ab3d-5abd05d845f0 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.684903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.686074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.686363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.686937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.687976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=595036af48cc4f96a5426e8f0b6d9ccf) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:24.688807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:24.690938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75426a50-1f69-4212-98f5-f5911a4c2353 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:24.698520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 53/114] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/role_assignments?user.id=595036af48cc4f96a5426e8f0b6d9ccf&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:24.699938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41199-8d54-430f-8942-0bf6da99a046 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.700235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41199-8d54-430f-8942-0bf6da99a046 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.701230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.702019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.702311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.702652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.703697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:24.704012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:24.705481 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f54e4a-e25e-4744-896f-d58553f79f2e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:24.707944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea41199-8d54-430f-8942-0bf6da99a046 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.710451 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 54/115] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:24.711222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea41199-8d54-430f-8942-0bf6da99a046 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.721878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 62/116] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:24.723266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.723570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.723864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.759686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:24.760949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:24.825049 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:24.864146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea41199-8d54-430f-8942-0bf6da99a046 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:24.871775 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 63/117] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:24.877980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.924230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:24.925493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.926028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.926607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.942560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-c66cfd95-0a26-49f9-b554-0933bef4fbc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:24.951067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:24.952188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:24.953730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 55/118] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:24.955090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bad6-265a-4634-9172-df1b0e03aa43 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:24.955090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bad6-265a-4634-9172-df1b0e03aa43 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:24.955241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1bad6-265a-4634-9172-df1b0e03aa43 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:24.956620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 56/119] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:24.957397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1bad6-265a-4634-9172-df1b0e03aa43 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:24.976321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.024755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fb766-530f-47be-8a51-32f54a519b04 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.072836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.074283 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.075002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.075526 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.077713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.080845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.087236 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 64/120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.087236 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.104449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1bad6-265a-4634-9172-df1b0e03aa43 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.159598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.160939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.161359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.161757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.164875 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.179796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-db8e6b78-f38f-4f83-b5f8-f2cf66b08898 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:25.192533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:25.193793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:25.194804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 57/121] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:25.196514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.196887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.197354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.217509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:25.219210 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:25.233596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f723f11-058e-40fd-8c03-fe0c3461c716 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.270612 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.285279 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.286623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.287220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.287726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.288746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:25.289753 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.291646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.304601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 65/122] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.305514 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.308921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 58/123] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:25.310060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.440874 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab52b7f-e9e1-425b-a187-b5cb368b3103 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.441712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05d07a-5ce2-404f-b14d-548504f11aa4 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.472896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.489190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.490079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.490377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.490667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.490955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.491899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.492165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.492690 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.493194 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.493425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.494390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:25.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.500649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 66/124] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.500649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.500910 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.513550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 59/125] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.515321 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.569732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.623235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb30e8-8e0d-4960-91d4-817d69d7549b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.653066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84386d-cdc4-4ac7-90bf-333262785c86 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.676176 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.677566 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.678041 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.678508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.680563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.682739 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.685339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 67/126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.686305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.704190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.705177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.705601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.705996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.719351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:25.728625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:25.729725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:25.730653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 60/127] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:25.731514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.787893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.819320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d66f2-4ded-42a8-80a7-05f4b349c461 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.852656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:25.864727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccef77c-5846-4b1f-8d84-7e0d68acc06e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:25.865829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.866974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.867372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.867765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/users/a86a18b340304607b0aafdfa7ad74e79/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.870315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, user_id=a86a18b340304607b0aafdfa7ad74e79, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.872169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.896426 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 68/128] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:25 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/users/a86a18b340304607b0aafdfa7ad74e79/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:25.896426 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:25.936943 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:25.941105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:25.941859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:25.942208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:25.946170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:25.949823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:25.955111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 61/129] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:25.957621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.097172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f0613-db3f-47b2-9bc1-c79d3f969a31 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.142254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.143269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.143917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.143917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.145210 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.146640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.148679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 69/130] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:26.149516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.165300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d2fbf-49a4-4c23-9806-c25e7aa4b5fa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.182840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:26.223609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:26.230191 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.231332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.231632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.231926 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.232808 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:26.233129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.235199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.255857 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 62/131] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:26.256732 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.278970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b861e-70a9-41f8-ac36-308d4d088edd None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.321088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.322105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.322437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.322682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.324027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.325696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.327478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 70/132] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:26 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:26.328544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.443100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d867718-46c7-43cb-b23f-fbd4f7debeab None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.485135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:26.488394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.489944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.490456 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.490963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.492902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.494874 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.496845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 71/133] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:26 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:26.497781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.550552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbce63c-a3b2-4c4f-bf09-95a568db6c25 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.618712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3e814-2936-466f-a4f2-134994280d45 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.656660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.657697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.658001 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.658299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/users/50716494befb4be495382d33e183829c/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.660941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, user_id=50716494befb4be495382d33e183829c, role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.662468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.674171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.675100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.675418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.675799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/users/50716494befb4be495382d33e183829c/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.678199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, user_id=50716494befb4be495382d33e183829c, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.679684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe49fa4-acdc-4ab5-bf82-80d5fbd4dbb3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.680311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 72/134] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:26 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/users/50716494befb4be495382d33e183829c/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:26.681355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.739150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 63/135] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:26 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/users/50716494befb4be495382d33e183829c/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:26.784892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:26.829366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562411-415e-4856-a8db-c139032c9a0c None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.829748 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562411-415e-4856-a8db-c139032c9a0c None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.829949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11562411-415e-4856-a8db-c139032c9a0c None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.836540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 64/136] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:26.839966 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea803e3-bb0c-4228-a9fa-453860239a8b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:26.843376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.844223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.852613 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.896811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:26.897726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:26.898259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:26.898558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] PATH_INFO: `/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:26.899951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] RBAC: Authorizing `identity:get_project(project_id=e04b1cdfbeb6460bb579d7328e46ce04)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:26.901846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:26.904053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 73/137] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:26 2022] GET /identity/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:26.909408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:26.909723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:26.910630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:27.026961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:27.134940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 65/138] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:27.143452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b383-d597-4cf0-8f34-5b79813e418b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:27.169795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:27.181501 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:27.233150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:27.234135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.234919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.235213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] PATH_INFO: `/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04/users/50716494befb4be495382d33e183829c/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.237529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] RBAC: Authorizing `identity:create_grant(project_id=e04b1cdfbeb6460bb579d7328e46ce04, user_id=50716494befb4be495382d33e183829c, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:27.239033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbbb6e-ac53-499d-a7dc-fd290d83ee2a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:27.268925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 74/139] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:26 2022] PUT /identity/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04/users/50716494befb4be495382d33e183829c/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:24:27.399524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7aff5-687a-436f-a344-ec3882b67ead None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:27.403410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4952-2e83-40d4-ba91-bd6b35ff075f None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.403741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4952-2e83-40d4-ba91-bd6b35ff075f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.403974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b4952-2e83-40d4-ba91-bd6b35ff075f None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.405468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 75/140] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:27.412256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.412615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.412841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.445133 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:27.446403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:27.528717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:27.529555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.529848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.530135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.532558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:27.534271 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:27.536235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 66/141] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:27.548110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:27.605223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 76/142] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:27.630392 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:27.633381 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:27.763382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b051e6f-54a7-41de-9b12-24a74e4926f5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:27.773895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:27.791214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a438f-0de0-47c4-a437-d6b9867964c4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:27.811547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:27.812662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.813185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.813802 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.837923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:27.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:27.850487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:27.851488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 77/143] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:27.868435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:27.886595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:27.887300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:27.887597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:27.887890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:27.888676 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:27.890216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:27.967029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 67/144] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:27 2022] POST /identity/v3/users => generated 283 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:24:28.045392 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e520e-b984-48b9-b610-1cff8d3e13e1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:28.092935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:28.094156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.094463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.094808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.095629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:28.095952 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:28.097443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:28.108138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 78/145] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:28.116651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:28.284773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:28.296285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beadcca-3ffb-4484-9930-c7cbf4b4d4a5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:28.351276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:28.352185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.352502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.352794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] PATH_INFO: `/v3/users/595036af48cc4f96a5426e8f0b6d9ccf` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.354030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] RBAC: Authorizing `identity:get_user(user_id=595036af48cc4f96a5426e8f0b6d9ccf)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:28.355521 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:28.357397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 68/146] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:28 2022] GET /identity/v3/users/595036af48cc4f96a5426e8f0b6d9ccf => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:28.365692 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:28.435331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:28.521423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3525c-279e-482a-8a87-b344c6a036a0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:28.561637 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:28.562474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.562754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.563034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.564570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:28.566028 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db7013-dbef-4a48-8a4a-1fa97b0b7c37 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:28.568056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 79/147] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:28 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:28.573539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:28.715719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025b122-9b2f-403f-862f-404e2fb7349b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:28.764044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:28.765931 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.766509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.767020 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/users/595036af48cc4f96a5426e8f0b6d9ccf/roles/f589e1251c7e4ffb903f695560d349a8` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.773847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, user_id=595036af48cc4f96a5426e8f0b6d9ccf, role_id=f589e1251c7e4ffb903f695560d349a8)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:28.775660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6bbea-095c-4cb3-8fc2-b28e100ef385 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:28.797900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 69/148] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:28 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/users/595036af48cc4f96a5426e8f0b6d9ccf/roles/f589e1251c7e4ffb903f695560d349a8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:28.907099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cab-523f-4fb1-b87c-fd9e14588803 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.907464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cab-523f-4fb1-b87c-fd9e14588803 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.907752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11696cab-523f-4fb1-b87c-fd9e14588803 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.909421 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 80/149] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:28.915295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:28.915862 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:28.916363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:28.940997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:28.942669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:28.964876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:29.046220 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 70/150] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:29.083949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696cab-523f-4fb1-b87c-fd9e14588803 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:29.222794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:29.227377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11696cab-523f-4fb1-b87c-fd9e14588803 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:29.270702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:29.271524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:29.271875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:29.272163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:29.284658 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:29.293082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:29.294138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:29.295075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 81/151] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:29.303410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:29.426856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c5b5a-b050-4b54-aa69-bf88761f7520 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:29.478044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:29.479134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:29.479733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:29.480474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:29.482016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:29.482777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:29.485391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:29.501997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 71/152] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:29.509624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:29.645403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfeead-bade-4d7f-91a5-edf658e70465 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:29.686825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:29.688959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:29.689249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:29.689528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] PATH_INFO: `/v3/users/a86a18b340304607b0aafdfa7ad74e79` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:29.693240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] RBAC: Authorizing `identity:get_user(user_id=a86a18b340304607b0aafdfa7ad74e79)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:29.694868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:29.696873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 82/153] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:29 2022] GET /identity/v3/users/a86a18b340304607b0aafdfa7ad74e79 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:29.702246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:29.722050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:29.871473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa6210-9a93-4176-ae5b-4b189181f2bf None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:29.916001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:29.916708 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:29.916999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:29.917280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.020201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:30.021768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:30.040569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:30.151868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 72/154] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:30.159180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:30.220791 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:30.325422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b62a5-144b-4ebb-b04f-6a8e8e28c945 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:30.375761 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:30.376760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:30.377044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:30.377322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.378593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:30.380186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af010cd8-c0b1-457a-9559-857ea63b3e05 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:30.382270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 83/155] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:30 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:30.388159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:30.498848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709caab-95b2-4b0e-82c2-9013dbb0d408 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:30.499177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709caab-95b2-4b0e-82c2-9013dbb0d408 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:30.499351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709caab-95b2-4b0e-82c2-9013dbb0d408 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.501189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 84/156] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:30.518785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:30.519124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:30.519357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.545810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:30.547124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:30.559552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c506-3588-4534-8c4d-40ea115eb15c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:30.581592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:30.659516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 85/157] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:30.677733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:30.678930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:30.679450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:30.679983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.681275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=a86a18b340304607b0aafdfa7ad74e79) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:30.681603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:30.684131 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:30.692685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 73/158] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:30 2022] GET /identity/v3/role_assignments?user.id=a86a18b340304607b0aafdfa7ad74e79&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:30.706770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:30.747560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:30.847536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bb7f0-6396-4ca2-a39c-853766b5eeb2 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:30.908894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:30.909744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:30.910025 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:30.910301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:30.924038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0e5e1d74-98e0-423d-8f8d-b5b9ee4717b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:30.940251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:30.941412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:30.942386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 86/159] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:30.952766 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:31.049123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d83fa0-1784-4fa9-8f9f-b730277f5304 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.049483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d83fa0-1784-4fa9-8f9f-b730277f5304 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.049697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d83fa0-1784-4fa9-8f9f-b730277f5304 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.051666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 87/160] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:31.057693 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.058050 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.058260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.089158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:31.090363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:31.160617 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4adf4b-4e24-4381-81c2-f6a83f22c3a7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:31.210507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 88/161] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 20:24:31.236964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:31.237765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.238054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.238340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.239262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:31.239581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:31.241254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:31.248929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:31.253673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 74/162] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:31.259425 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:31.278911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:31.355236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:31.462879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c06a2-67a1-465b-8de9-6c19c7bd1c95 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:31.470595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbe8f7-fd2a-44ba-811d-ad1d8b417311 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:31.520676 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:31.522086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.522616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.523110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.525889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:31.527724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c65738-8397-41ba-b1ba-3637d32b656d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:31.530011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 75/163] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:31 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:31.532311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b8cb0-664b-4d06-bb79-76e6f165a72f None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.532885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b8cb0-664b-4d06-bb79-76e6f165a72f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.533409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b8cb0-664b-4d06-bb79-76e6f165a72f None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.535353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 76/164] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:31.539972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b8cb0-664b-4d06-bb79-76e6f165a72f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:31.545520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:31.546346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.546777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.547085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.577506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2f135b0c-a30b-4c91-95eb-7972f643386a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:31.585736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:31.586879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:31.587801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 89/165] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:31.593159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.594162 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.594610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.625431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:31.626625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:31.753404 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b8cb0-664b-4d06-bb79-76e6f165a72f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:31.758059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:31.778070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 90/166] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:31.779018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:31.784652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:31.807722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:31.811938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:31.813349 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:31.814092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:31.972559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c5e00-c9ab-4aa1-a963-0b3480a4be26 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.037426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.038856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.046224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.047979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.048521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.049528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.050555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:32.051097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.052824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.073463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 91/167] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:32.075357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.198867 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:32.252558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757deb5-6a45-4326-b75e-2e796fc23e4e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.301148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.302322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.302840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.303362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.309386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 77/168] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:32.310266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.317381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:32.319494 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:32.329584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:32.330726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:32.331678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:32.335920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 92/169] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:32.337203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.504531 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0870e8-6787-491c-aba5-47ea8d83ec07 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.509333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787f01-e9ad-4743-bd6b-d62b3012a05e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.548764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:32.558531 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.559349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.559638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.559916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.561197 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.563768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.565813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 93/170] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:32.570578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.571681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.573019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.573475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.573930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.575522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.577282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.579714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 78/171] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:32.580905 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.756849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:32.793999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175585-799d-41d0-a4da-f32a8847d71e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.814871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2ad8-0d1a-4a87-9b27-2b51ea0ed862 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:32.838046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.839205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.839743 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.840251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.841265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:32.841847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.844096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.855461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 94/172] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:32.857080 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.915907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:32.915907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:32.915907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:32.916312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:32.917107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=50716494befb4be495382d33e183829c) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:32.917430 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:32.918933 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:32.931653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 79/173] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 608 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:32.932553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:32.940913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.035939 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb7982-36eb-4784-9b08-2424450e5649 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:33.103099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:33.104239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.105429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.106115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.211218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6621d8-b973-4cee-ab0b-808db5fe952e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:33.216812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:33.218393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:33.261893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:33.263687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.264198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.264727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] PATH_INFO: `/v3/users/50716494befb4be495382d33e183829c` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.269164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] RBAC: Authorizing `identity:get_user(user_id=50716494befb4be495382d33e183829c)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:33.270949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edab17-8f84-4352-998f-cf61834f67b1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:33.276573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 80/174] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/users/50716494befb4be495382d33e183829c => generated 275 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:33.278507 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef634f7-e0fa-4c7a-9097-562841e59da4 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.279081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef634f7-e0fa-4c7a-9097-562841e59da4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.279591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef634f7-e0fa-4c7a-9097-562841e59da4 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.281358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 81/175] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:33.287577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.288169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.288703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.301083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.310317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.320008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:33.321272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:33.403662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 95/176] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:33.404679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:33.456555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 82/177] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:33.457577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:33.613159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47da2-92b0-418a-947a-3d8d299177ff None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:33.639045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095aeae-b5e6-45a0-b813-5514244fc5c9 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:33.659863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:33.660493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.660794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.661063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.687834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:33.689204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.689497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.689780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.691620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:33.692171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.692822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:33.694901 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 83/178] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:33 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:33.695624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:33.699177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.760281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:33.761781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316281-33e0-4c48-8f98-06b0a04f54e2 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:33.817874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738e844-040c-40df-9223-a8c72b4b92cb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:33.854484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:33.855313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.855605 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.855887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.868664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:33.871069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 96/179] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:33.872623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd139b3-e529-4a43-9c00-801c53fad43c None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.872914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd139b3-e529-4a43-9c00-801c53fad43c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.873198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd139b3-e529-4a43-9c00-801c53fad43c None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.875256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 97/180] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:33.876433 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c830a56-92a8-46fe-a992-94a95f6a07a0 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.876727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c830a56-92a8-46fe-a992-94a95f6a07a0 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.877077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c830a56-92a8-46fe-a992-94a95f6a07a0 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.878464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 98/181] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6f0b4aae-13a9-42aa-99a4-8bd2ff81a6d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:33.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:33.881193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:33.882134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:33.882850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c830a56-92a8-46fe-a992-94a95f6a07a0 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:33.882937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 84/182] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:33.882937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5679a89-aafc-4d5b-9410-3c172b05b931 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:33.883185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5679a89-aafc-4d5b-9410-3c172b05b931 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:33.883943 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5679a89-aafc-4d5b-9410-3c172b05b931 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:33.884793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 85/183] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:33.889243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5679a89-aafc-4d5b-9410-3c172b05b931 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:33.956079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:33.976962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.005307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c830a56-92a8-46fe-a992-94a95f6a07a0 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.019406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5679a89-aafc-4d5b-9410-3c172b05b931 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.041438 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.042103 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.042397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.042681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.043525 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=50716494befb4be495382d33e183829c) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:34.043846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.045326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0e5c-84b0-47c0-8511-54f65cc614f3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.053317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.054199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.054482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.054757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] PATH_INFO: `/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.055970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] RBAC: Authorizing `identity:get_project(project_id=e04b1cdfbeb6460bb579d7328e46ce04)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.057488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ed303-f132-4857-aaec-f6949e4472dd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.059470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 86/184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:33 2022] GET /identity/v3/projects/e04b1cdfbeb6460bb579d7328e46ce04 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.059897 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 99/185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:33 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.061305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0dc6a-6cfb-4be6-b291-d9615bc47259 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.061457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.061742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.062070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.062927 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0dc6a-6cfb-4be6-b291-d9615bc47259 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.063386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0dc6a-6cfb-4be6-b291-d9615bc47259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.082725 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:34.084009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:34.086120 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-27a0dc6a-6cfb-4be6-b291-d9615bc47259 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:34.087357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-27a0dc6a-6cfb-4be6-b291-d9615bc47259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:34.169693 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 100/186] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:34.177573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.180081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 87/187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:34.190320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.190883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.191388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.224529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:34.225707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:34.244524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.298001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.311320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 88/188] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:34.313000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.315355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172426d-3d2c-4a2d-bb64-ef4ca9b0247a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.362917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.363600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.363881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.364157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.365263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:34.365263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.366658 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.376499 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 101/189] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.377382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.427987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b750f2a-5603-462d-bef6-4b1fb6f51ceb None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.464731 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.465406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.465697 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.466010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.466866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=e04b1cdfbeb6460bb579d7328e46ce04, user.id=50716494befb4be495382d33e183829c) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:34.467186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.468919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.475449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 89/190] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/role_assignments?user.id=50716494befb4be495382d33e183829c&scope.project.id=e04b1cdfbeb6460bb579d7328e46ce04&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.476257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.497310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf6501-e6ce-454d-af1e-d50d8c8b3cf4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.513928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.541500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.542447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.542812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.543208 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.554142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.565252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:34.573452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:34.574589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:34.575517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 102/191] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:34.576318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.616551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6245d-8546-45b6-9c29-f7091d09fa1f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.660575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.661538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.661882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.662079 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.675275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:34.684386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:34.685869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:34.687819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 90/192] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:34.689785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.711018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac9135-a336-482f-844d-0f2ff883b9cf None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.759728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.760583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.760879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.761163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.762390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.763859 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.766364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 103/193] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.767230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.811681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.815343 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be981ea-7e28-4730-aaa3-3dcc9a7f59fd None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.830175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:34.852724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.853615 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.853899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.854176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] PATH_INFO: `/v3/users/595036af48cc4f96a5426e8f0b6d9ccf` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.855385 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] RBAC: Authorizing `identity:get_user(user_id=595036af48cc4f96a5426e8f0b6d9ccf)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.856913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.858730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 91/194] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/users/595036af48cc4f96a5426e8f0b6d9ccf => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.859980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.882979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c9313-a072-4737-8ed1-11250dadafc5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:34.919239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:34.919922 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:34.920216 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:34.920540 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:34.921534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:34.921924 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:34.923323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:34.931593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 104/195] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:34.932421 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:34.965977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e42204-b218-4731-8652-4f6b2e4f07cd None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.001436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.002123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.002441 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.002674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.003442 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:35.003755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.005204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.014060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 92/196] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:35.014824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.039618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb0bb-ff83-4ef4-a479-83aa1f1300ef None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.041194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.064555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.075907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.076730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.076919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.077201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.078464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.079930 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.107550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 105/197] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:34 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:35.107550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.141101 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f621-ab9b-4f21-a9fb-d0355352f317 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.189564 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.190252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.190534 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.190817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.230419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861fef5-870b-4be0-bef2-21ef6ab06de1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.271368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.272186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.272508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.272797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.273674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.275719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.289568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.291073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.292921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 106/198] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:35.293724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.321482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.341957 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.422976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 93/199] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:35.423739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.439404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c28cb7-bdca-4a39-a826-7041c4f4d442 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.482368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.483432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.483721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.484030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.486111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.487755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.489798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 107/200] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:35.493222 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.558981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347ff50-889a-4be8-8e9a-52065962f8b6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.601713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def43939-e034-495c-967b-760e5a602e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.602886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def43939-e034-495c-967b-760e5a602e88 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.603398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def43939-e034-495c-967b-760e5a602e88 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.603923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def43939-e034-495c-967b-760e5a602e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.609813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.625424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.630884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b4ce-1539-4aa6-97b5-3f42fc6ef658 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.683489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.684821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.685356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.685951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.688299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.690422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.697683 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def43939-e034-495c-967b-760e5a602e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.697683 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def43939-e034-495c-967b-760e5a602e88 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:35.698209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 108/201] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:35.699439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.827514 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c54a19-7d5d-45ec-9069-27cf3bdcb187 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.843010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 94/202] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:35.847854 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def43939-e034-495c-967b-760e5a602e88 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:35.874938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:35.876249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:35.876902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:35.877419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:35.902970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.905291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:35.963568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:35.964086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def43939-e034-495c-967b-760e5a602e88 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:35.965848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.004232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:36.004949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:36.005239 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:36.005705 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:36.006687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, scope.project.id=c27ff1b3891041938db6c68e900e9136, user.id=595036af48cc4f96a5426e8f0b6d9ccf) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:36.007011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:36.008529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.014891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 95/203] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/role_assignments?user.id=595036af48cc4f96a5426e8f0b6d9ccf&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:36.016358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:36.090908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 109/204] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:36.097882 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:36.184710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2d3e-b1bf-4d52-92a5-d579db405a01 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:36.228805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:36.245709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:36.261074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:36.273048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:36.273409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:36.273649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:36.275184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:36.276832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.279113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 96/205] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:36 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:36.286051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:36.311809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba9baf-5037-46a1-8c3e-92b7cebe69fa None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:36.370899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:36.371714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:36.372005 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:36.372284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:36.378315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:36.379813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.381893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 110/206] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:36 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:36.390225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:36.457429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0977-76ef-4a5c-9133-07a92f00071d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:36.498785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:36.500279 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:36.502170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:36.502479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:36.503416 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=2309e04932b44854aac2396bb1241de9) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:36.503701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:36.505317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee1237-4001-44b4-92fa-b47a92e1492e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.512613 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 97/207] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:36 2022] GET /identity/v3/role_assignments?user.id=2309e04932b44854aac2396bb1241de9&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:36.530307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183db38a-d925-4472-b5bb-3cbff2065ae0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:36.556863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:36.577654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:36.578484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:36.578785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:36.579074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:36.579965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=2309e04932b44854aac2396bb1241de9) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:36.580292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:36.581926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8ee1f-e13b-4a40-b8d9-75e20ad3652e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:36.589112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 111/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:36 2022] GET /identity/v3/role_assignments?user.id=2309e04932b44854aac2396bb1241de9&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:37.614936 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364f9f-1854-47cd-87d8-cff8516e54c4 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:37.615475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364f9f-1854-47cd-87d8-cff8516e54c4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:37.615475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44364f9f-1854-47cd-87d8-cff8516e54c4 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:37.617072 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 98/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:37.622516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:37.623388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:37.623878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:37.641150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:37.642664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:37.721832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb136-ca23-4c04-891a-d0475dd43198 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:37.722202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb136-ca23-4c04-891a-d0475dd43198 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:37.722427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cb136-ca23-4c04-891a-d0475dd43198 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:37.723947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 99/210] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:37.729389 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:37.729686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:37.729974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:37.730942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 112/211] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:37.752807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:37.754373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:37.754781 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:37.807257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:37.851186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 100/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:37.877656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddfc50-32a0-4811-88cf-46b3c08e3318 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:37.881094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:37.894317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:37.922306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:37.923551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:37.924066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:37.924597 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:37.938885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:37.949682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:37.950789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:37.951726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:37.953809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 113/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:37.960688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.007411 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8dcac-4f12-493e-bbda-c86a68aab000 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.046081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.047473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.048002 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.048587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.050018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.051854 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.058649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 101/214] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:38.059891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.071477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51e599-1152-4d19-8657-ad1b3e29f479 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.113181 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.113450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:38.115009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.115548 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.116195 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.117945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:38.118570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.120953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.133209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 114/215] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:38.138370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.185111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:38.197694 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60fa3-9e4b-47f0-a059-b3414837870a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.244870 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.245980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.246484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.247049 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.248013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.250430 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.274646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d4bf9-1cd5-4b43-b119-360a4e61cbc6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.284896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 102/216] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:38 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:38.322573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.323490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.323777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.324062 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] PATH_INFO: `/v3/users/6edca12a10af4fb39f2227dd74eb7872` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.340318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] RBAC: Authorizing `identity:get_user(user_id=6edca12a10af4fb39f2227dd74eb7872)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.341898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.344219 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 115/217] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:38 2022] GET /identity/v3/users/6edca12a10af4fb39f2227dd74eb7872 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:38.350153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.467581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:38.476035 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98a9af7-4f73-4af6-9c18-427ddb6f4607 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.519726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.520499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.520767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.521050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.602086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.603651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d023f-a4ea-4c67-8ef9-d7af6fc12f13 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.733188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:38.747776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 103/218] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:38.755484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.880693 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:38.890706 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ac350-91e2-4559-be7f-d228f421ad60 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:38.933047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:38.933938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.934268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.934552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.935783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:38.937397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:38.939695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 116/219] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:38 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:38.940638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23028be9-4cc3-437d-aa22-069c3bb77964 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.941171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23028be9-4cc3-437d-aa22-069c3bb77964 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.941673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23028be9-4cc3-437d-aa22-069c3bb77964 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.943294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 104/220] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:38.945587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:38.949487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98b172-d88f-437b-9c3b-ba5b96374670 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:38.950110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98b172-d88f-437b-9c3b-ba5b96374670 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:38.950628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98b172-d88f-437b-9c3b-ba5b96374670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:38.969921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ce98b172-d88f-437b-9c3b-ba5b96374670 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:38.971524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ce98b172-d88f-437b-9c3b-ba5b96374670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:39.059789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61aed4-ee51-4a21-a993-ab0d4714a65f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.074154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 105/221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:39.082155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d9074-4f46-4dea-a277-4376c86dd436 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.082763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d9074-4f46-4dea-a277-4376c86dd436 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.083264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d9074-4f46-4dea-a277-4376c86dd436 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.085245 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 106/222] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:39.090516 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.091075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.091706 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.095965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.100457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.100457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.100457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.100457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=6edca12a10af4fb39f2227dd74eb7872) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:39.100457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:39.100883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:39.106775 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 117/223] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:38 2022] GET /identity/v3/role_assignments?user.id=6edca12a10af4fb39f2227dd74eb7872&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:39.113512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.123350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:39.125215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:39.202364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:39.220949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 107/224] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:39.242460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.324863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:39.348863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71d111-094c-4c7e-ba17-9daa8113186f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.392389 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddd95-2143-44d6-9f30-8f327fa6c35c None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.393011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.394108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.394649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.395279 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.407862 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:39.416309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:39.417465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:39.418395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:39.421135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 118/225] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:39.428230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.435491 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.435995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.436285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.436591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.452163 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:39.463017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:39.464193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:39.465453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:39.468902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 108/226] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:39.472525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.551561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:39.568625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f31f16-7560-4791-b536-b999ea772e7d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.613630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.614493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.614885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.615266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.616128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:39.616779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:39.618090 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:39.620123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc637e-edcc-420e-9cbe-2b9ef5125c4d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.628749 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 119/227] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:39.634800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.642012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:39.667963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.669264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.669734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.670209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.672481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:39.672963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:39.674555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:39.692085 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 109/228] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:39.701480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.787925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca873db-e70f-4ae0-8720-e254ab4750ea None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:39.831517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:39.832999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:39.833529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:39.834037 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:39.835596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:39.837745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:39.840622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 120/229] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:39.853477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:39.893018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:39.921092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8a0e3-8d3f-4d6f-a438-f6d038ca105d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:40.001672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb77c0-015b-4bed-ab1c-4ba2c700315f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:40.007028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:40.007938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:40.008229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:40.008539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:40.009792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:40.011272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:40.013002 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 110/230] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:40.019419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:40.082572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:40.083485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:40.083778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:40.084061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:40.086349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:40.087872 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:40.092821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 121/231] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:39 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:40.097472 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:40.109354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:40.181342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac91c2-a820-4b69-86f4-68b268778f73 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:40.257608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:40.258513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:40.258803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:40.259084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:40.268046 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:40.269631 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:40.271485 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 111/232] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:40 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:40.277235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:40.292346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:40.326173 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b2d4b-91c3-41a8-9e73-c7ddc37cc0b9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:40.400364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:40.402200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:40.402875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:40.406071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/users/2309e04932b44854aac2396bb1241de9/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:40.409682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, user_id=2309e04932b44854aac2396bb1241de9, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:40.412562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71ec0-a2f2-4ad1-a4da-ac923ac1cd9d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:40.440587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 122/233] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:40 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/users/2309e04932b44854aac2396bb1241de9/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:40.458940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b81df-6bbc-4c82-ab63-1d3850979ec4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:40.476099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:40.501143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:40.502472 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:40.502965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:40.503465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/users/2309e04932b44854aac2396bb1241de9/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:40.506027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, user_id=2309e04932b44854aac2396bb1241de9, role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:40.507754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478fe0a-689b-4f73-94ab-67dfa3c04556 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:40.527611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 112/234] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:40 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/users/2309e04932b44854aac2396bb1241de9/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:41.313149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72f690-714f-4b69-97cc-e1b6349e4589 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.314077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72f690-714f-4b69-97cc-e1b6349e4589 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.314077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72f690-714f-4b69-97cc-e1b6349e4589 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.315454 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 123/235] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:41.321520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa761c-d05e-405b-8081-d57d6091a2dc None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.322060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa761c-d05e-405b-8081-d57d6091a2dc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.322325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa761c-d05e-405b-8081-d57d6091a2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.341028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-40aa761c-d05e-405b-8081-d57d6091a2dc None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:41.342244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-40aa761c-d05e-405b-8081-d57d6091a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:41.421815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 113/236] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:41.445821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72f690-714f-4b69-97cc-e1b6349e4589 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:41.551459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dcc2d-b814-4c45-a504-9968d2c003ab None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.551745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dcc2d-b814-4c45-a504-9968d2c003ab None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.552029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dcc2d-b814-4c45-a504-9968d2c003ab None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.553470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 114/237] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:41.558838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.559204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.559489 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.589385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72f690-714f-4b69-97cc-e1b6349e4589 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:41.601314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:41.602524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:41.657881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:41.658707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.658995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.659274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.680646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:41.689288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:41.690405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:41.691353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 124/238] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:41.697491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:41.719824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:41.751859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 115/239] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:41.790695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:41.828886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:41.865833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b1fc-f8f3-466a-beb8-16be0d6d8ee6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:41.910652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44432-811c-4f46-8d17-811ad289fa68 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:41.946081 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:41.946908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.947204 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.947495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.948387 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:41.948845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:41.949802 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563009-82b0-40d1-b877-906444823393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:41.950248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:41.950607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563009-82b0-40d1-b877-906444823393 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:41.950901 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563009-82b0-40d1-b877-906444823393 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:41.951179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7563009-82b0-40d1-b877-906444823393 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:41.952576 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563009-82b0-40d1-b877-906444823393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:41.954579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7563009-82b0-40d1-b877-906444823393 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:41.956610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 116/240] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:41.961581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563009-82b0-40d1-b877-906444823393 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:41.973853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 125/241] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:41.995235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.077004 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:42.109218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:42.113333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7563009-82b0-40d1-b877-906444823393 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.131726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a0bab-a5bd-406c-97ef-aba43d8684f0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.168025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.169214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.169689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.170087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.170296 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.171209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:42.171899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.172242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.172645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] PATH_INFO: `/v3/users/f0cd4643c31446a0990c4870361f7151` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.173969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d1767-d0ea-4e14-a478-10c140a33ffa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:42.195755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] RBAC: Authorizing `identity:get_user(user_id=f0cd4643c31446a0990c4870361f7151)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:42.197940 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b3df8-9c88-4423-bf47-a800297f7173 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:42.199880 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 126/242] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:41 2022] GET /identity/v3/users/f0cd4643c31446a0990c4870361f7151 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:42.201116 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f38f8-86ce-46ba-a8cb-fa4011f1f49a None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.201415 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f38f8-86ce-46ba-a8cb-fa4011f1f49a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.201697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f38f8-86ce-46ba-a8cb-fa4011f1f49a None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.209394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 127/244] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:42.209394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f38f8-86ce-46ba-a8cb-fa4011f1f49a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.209881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 117/244] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:41 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:24:42.211594 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.212047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.212751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.257929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:42.260171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:42.427111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 118/245] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:42.442336 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f38f8-86ce-46ba-a8cb-fa4011f1f49a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.474164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.484145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:42.505276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.505955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.506244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.506525 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.616286 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:42.617052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:42.618178 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:42.638378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746893-c643-4144-ac93-d90c2c66181f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.694266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.696059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.696771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.697273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.720067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:42.728759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:42.729827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:42.730765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:42.734395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 119/246] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:42.740981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.762591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 128/247] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:42.768144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.802031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:42.895079 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463247e-2be1-4812-8a32-2e76fd186727 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.909300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ff41-1623-422c-b34b-6399db658b39 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:42.944352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.945104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.945497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.945905 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.946787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:42.947209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:42.948752 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:42.960158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:42.961345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:42.961783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:42.962123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 120/248] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:42.962364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:42.963813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:42.965616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:42.967914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 129/249] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:42.969780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.975299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:42.984633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:43.091439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:43.101214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc221664-d796-41a4-90b9-f39c04e8542a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:43.103823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ad6ad-ff32-47c3-bc18-fc16f533865f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:43.147083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:43.147437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.147732 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.148017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] PATH_INFO: `/v3/users/6edca12a10af4fb39f2227dd74eb7872` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.149331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] RBAC: Authorizing `identity:get_user(user_id=6edca12a10af4fb39f2227dd74eb7872)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:43.150527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:43.150983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:43.151803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.152307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.152816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.153225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 121/250] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/users/6edca12a10af4fb39f2227dd74eb7872 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:43.154112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=f0cd4643c31446a0990c4870361f7151) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:43.154706 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:43.156728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:43.158847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:43.163979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 130/251] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:42 2022] GET /identity/v3/role_assignments?user.id=f0cd4643c31446a0990c4870361f7151&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:43.253769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:43.280891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86383edc-817c-402f-b65e-58becd3a96cb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:43.321735 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:43.322656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.322950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.323242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.324664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:43.326193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708a687-c748-4fca-a068-0ac5b7617094 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:43.329258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 122/252] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:43 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:43.335388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:43.510075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9158b93-1dd6-445e-a98c-2e79d77f4057 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.511076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9158b93-1dd6-445e-a98c-2e79d77f4057 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.511595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9158b93-1dd6-445e-a98c-2e79d77f4057 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.513367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 123/253] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:43.519826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e97f6-9b47-4c42-988c-2572e35dd0f5 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.520664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e97f6-9b47-4c42-988c-2572e35dd0f5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.521145 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e97f6-9b47-4c42-988c-2572e35dd0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.536438 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41462f-6614-477a-bf70-8416714764cc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:43.554378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c42e97f6-9b47-4c42-988c-2572e35dd0f5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:43.555937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c42e97f6-9b47-4c42-988c-2572e35dd0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:43.603277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:43.604571 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.605087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.605563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/users/6edca12a10af4fb39f2227dd74eb7872/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.613109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, user_id=6edca12a10af4fb39f2227dd74eb7872, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:43.614613 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:43.644328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 131/254] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:43 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/users/6edca12a10af4fb39f2227dd74eb7872/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:43.681118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 124/255] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:43.702266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:43.716274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c832-1d32-420c-92cf-ac8e9c96f709 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.717244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c832-1d32-420c-92cf-ac8e9c96f709 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.717244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c832-1d32-420c-92cf-ac8e9c96f709 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.721928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 125/256] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:43 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:43.728272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:43.728621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:43.728878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:43.742270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:43.766453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:43.767599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:43.903962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 126/257] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:43.936342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:43.941504 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:43.954537 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598e2a8-949f-45a8-ab85-a4678e6920b7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.005220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.006058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.006349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.006636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.041142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:44.049914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:44.051058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:24:44.051991 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:44.052745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 132/258] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:44.056910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.075515 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d3e85-64f2-41c6-bf68-098718f4048d None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.114958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.115801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.116089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.116369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.130138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:44.138885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:44.139959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:44.141125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 127/259] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:44.148434 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.207359 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:44.248786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:44.260338 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3442bd-03c0-41d4-b9a2-756a3348830a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.303485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.304089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.304375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.304678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.305647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:44.306294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:44.307709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:44.318512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 133/260] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:44.324541 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.360665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e0f6c-0ba3-4fec-8d87-e4a50cb115f4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.447007 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3f42d-909c-4f7b-a84f-92d30566f96d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.465690 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.466393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.466687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.466973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.467753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:44.468099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:44.470276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:44.487898 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 128/261] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:44.493792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.495615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.496570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.496861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.497143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.498930 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:44.501215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:44.503023 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 134/262] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:44.517507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.546117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:44.767030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b75e0-5e50-4a4a-85d2-9d1c4728917f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.801413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f3399-6afd-478f-8f1f-faf5d2102599 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:44.823828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:44.853774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.855024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.855525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.856018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] PATH_INFO: `/v3/users/2309e04932b44854aac2396bb1241de9` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:44.861314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] RBAC: Authorizing `identity:get_user(user_id=2309e04932b44854aac2396bb1241de9)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:44.863128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:44.865461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 129/263] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/users/2309e04932b44854aac2396bb1241de9 => generated 283 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:44.872351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:44.899730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:44.900955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:44.902352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:44.903038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:45.085655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:45.087094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:45.100094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:45.104952 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f120f-916d-44d8-aede-2700d5395585 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:45.165282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:45.165998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:45.166296 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:45.166579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:45.273798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:45.284710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:45.286232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:45.418457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 135/264] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 901 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:45.429558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:45.443383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 130/265] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:24:45.448518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:45.637464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:45.652660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec7af0-c102-407b-90f5-fca627723ae7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:45.714662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cbfd8-d43d-44c7-9d31-2fc18cd970b7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:45.728815 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:45.733129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:45.737555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:45.737555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:45.737555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:45.737555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:45.739499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:45.741668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 131/266] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:45 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:45.747775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:45.796883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:45.797831 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:45.798122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:45.798408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:45.805321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:45.806820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:45.809044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 136/267] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:45 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:45.815341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:45.982649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09899239-57a0-4a8e-9cc6-5d72ee9ee65f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.025868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd8df2-cf0e-4878-8d24-6241262dda14 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.076364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.077104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.077423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.077706 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.078559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=2309e04932b44854aac2396bb1241de9) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:46.078872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:46.080344 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744aa72-a229-4d00-a07f-243d63604a76 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:46.090317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.091016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.091310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.091597 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.092478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=2309e04932b44854aac2396bb1241de9) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:46.094730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:46.094853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 132/268] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:45 2022] GET /identity/v3/role_assignments?user.id=2309e04932b44854aac2396bb1241de9&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 608 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:46.094938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:46.095476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b45e9b-5c9e-4e8c-a27a-c9085ba01e03 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.095770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b45e9b-5c9e-4e8c-a27a-c9085ba01e03 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.096052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b45e9b-5c9e-4e8c-a27a-c9085ba01e03 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.097371 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 133/269] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:46.101536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 137/270] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:45 2022] GET /identity/v3/role_assignments?user.id=2309e04932b44854aac2396bb1241de9&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:46.103846 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.104258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.104439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.131056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:46.132308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:46.176950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:46.227789 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 134/271] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:46.274576 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:46.330621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:46.408499 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d82651-f3dd-4cc4-af9d-b1e924a0394c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.449121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.449954 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.450242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.450524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.463155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2af25f48-f925-4881-ada5-239d82d89163 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:46.472144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:46.473293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:46.474232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 138/272] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:46.481603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:46.558110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e470-2634-4851-bbf8-089a3efb89c8 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.558762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e470-2634-4851-bbf8-089a3efb89c8 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.559276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589e470-2634-4851-bbf8-089a3efb89c8 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.560967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 139/273] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:24:46.568103 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.568613 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.568911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.594162 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:46.595379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:46.628114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470118f-9dcc-4322-927f-814d9ceb3be1 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.678689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.679793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.680559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.681073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.682120 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:46.682446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:46.684969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:46.693280 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 140/274] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:46.701050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 135/275] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:46.706412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:46.737879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:46.741034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:46.768389 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:46.843140 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f1ca9-e73e-4f9b-a444-ad57f3f5cdff None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.869287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef9984-5c33-4d84-8e0f-a1222cb99163 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:46.890316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.890575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.890795 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.891138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] PATH_INFO: `/v3/users/ad529be7caf74f8f86b26408bd3d020e` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.908655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] RBAC: Authorizing `identity:get_user(user_id=ad529be7caf74f8f86b26408bd3d020e)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:46.910103 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b813c54-a6b7-4514-a43a-90c76c81cdcf None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:46.912029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 141/276] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:46 2022] GET /identity/v3/users/ad529be7caf74f8f86b26408bd3d020e => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:46.913325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf4a1-b062-42d6-8a0b-7c8706e028ff None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.913685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf4a1-b062-42d6-8a0b-7c8706e028ff None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.913979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edf4a1-b062-42d6-8a0b-7c8706e028ff None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.915376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 142/277] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:46.916963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:46.918041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.918427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.918626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.919919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:46.920217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edf4a1-b062-42d6-8a0b-7c8706e028ff None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:46.921585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a782f-debd-4839-938a-97d84575531c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:46.923704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 136/278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:46.928621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:46.928621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:46.928621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:46.945972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:46.947197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:47.025600 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 137/279] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:47.027195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.049316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.058283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.072235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edf4a1-b062-42d6-8a0b-7c8706e028ff None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.114499 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.115179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.115475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.115763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.116641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, user.id=ad529be7caf74f8f86b26408bd3d020e, scope.system=all) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:47.116969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:47.118422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:47.125806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 143/280] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:24:46 2022] GET /identity/v3/role_assignments?user.id=ad529be7caf74f8f86b26408bd3d020e&scope.system=all&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:47.126647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.172723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092824c-c7e2-4626-affb-ce3b334c26f8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.216737 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.217440 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.217730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.218608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.219574 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:47.221147 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a8b61-6735-4a38-b46a-293e6da081b2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:47.257588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 138/281] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:24:47 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:47.259006 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da102-e248-4b51-9ff8-c2404c7fc2e8 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.259303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da102-e248-4b51-9ff8-c2404c7fc2e8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.259590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8da102-e248-4b51-9ff8-c2404c7fc2e8 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.264671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 139/282] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:47.267571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.267940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.268333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.282506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0074871-deb1-4700-a742-56335c652d33 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.289242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:47.290876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:47.334014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.336053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.336792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.337249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.356992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:47.368377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:47.369697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:47.370825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:47.373883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 144/283] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:47.381902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.396641 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 140/284] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:47.404888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.416787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.421196 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.525222 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5f7e9-e1f8-4389-a462-51b285c89186 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.570438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52098a13-fb28-45df-9e80-7cb2a87f3e80 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.585945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.586644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.587080 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.587387 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.588167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:47.588541 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:47.591570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:47.611097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 145/285] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:47.613653 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.617606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.619048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.619048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.619048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.636922 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:47.647135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:47.648305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:47.649320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:47.650015 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 141/286] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:47.658291 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.725729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.751699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:47.772475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564abf06-632a-4779-9876-f90d9e332422 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.801977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cd0ae4-0f2a-4b4b-abc9-11aeafa204a9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:47.812909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.814399 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.814920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.815403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] PATH_INFO: `/v3/users/f0cd4643c31446a0990c4870361f7151` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.816940 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] RBAC: Authorizing `identity:get_user(user_id=f0cd4643c31446a0990c4870361f7151)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:47.818844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:47.824615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 146/287] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/users/f0cd4643c31446a0990c4870361f7151 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:47.828539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.848105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:47.848877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:47.849182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:47.849502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:47.856424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:47.856424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:47.856424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:47.863483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 142/288] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:47.868778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:47.979009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff246-cb6d-4058-83b4-685d74dd0321 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:48.064441 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94abe82-1694-4e23-baa9-5ea9c50af3aa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:48.077143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:48.078655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:48.079234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:48.080331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:48.085638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:48.088642 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:48.091419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:48.096448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 147/289] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:48.105868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:48.110541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:48.111772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:48.112322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:48.112958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] PATH_INFO: `/v3/users/6edca12a10af4fb39f2227dd74eb7872` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:48.114407 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] RBAC: Authorizing `identity:get_user(user_id=6edca12a10af4fb39f2227dd74eb7872)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:48.116209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:48.118180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 143/290] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:47 2022] GET /identity/v3/users/6edca12a10af4fb39f2227dd74eb7872 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:48.124737 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:48.128835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:48.299798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0ad22-90b0-47f7-ab75-e8fdcbaaba01 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:48.349445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138386b-f377-42c6-8b8b-fa85c607eed0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:48.364710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:48.366043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:48.366554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:48.367050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:48.439307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:48.451822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:48.454256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:48.454710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:48.455190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:48.455480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:48.455763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/users/f0cd4643c31446a0990c4870361f7151/roles/f589e1251c7e4ffb903f695560d349a8` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:48.458086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, user_id=f0cd4643c31446a0990c4870361f7151, role_id=f589e1251c7e4ffb903f695560d349a8)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:48.459588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:48.491908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 148/291] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:24:48 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/users/f0cd4643c31446a0990c4870361f7151/roles/f589e1251c7e4ffb903f695560d349a8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:24:48.590423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 144/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:24:48.596560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:48.727986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:48.772171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa8685-9d0c-4f06-a81e-5607d29cff63 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:48.816601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:48.818063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:48.818643 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:48.819095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:48.821167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:48.823220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e23c1-a83d-432b-9e53-24d688c6f412 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:48.825412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 149/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:48 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:48.833141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:49.017675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:49.038549 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491e150-0050-43a8-a93e-05a254559b4b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:49.094871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:49.095553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:49.095820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:49.096104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:49.096986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=6edca12a10af4fb39f2227dd74eb7872) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:49.097306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:49.098782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4210bb-616d-4c50-a1e1-4b8e2f6a7c88 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:49.110783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 145/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:48 2022] GET /identity/v3/role_assignments?user.id=6edca12a10af4fb39f2227dd74eb7872&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:49.881833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a002d-7267-4212-a1cb-58dff63f6d67 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:49.882841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a002d-7267-4212-a1cb-58dff63f6d67 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:49.883361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a002d-7267-4212-a1cb-58dff63f6d67 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:49.885000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 150/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:49.891515 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b28e4-682b-4597-8a96-cd2fdeb11e93 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:49.891806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b28e4-682b-4597-8a96-cd2fdeb11e93 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:49.892294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b28e4-682b-4597-8a96-cd2fdeb11e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:49.912499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-d81b28e4-682b-4597-8a96-cd2fdeb11e93 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:49.913976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-d81b28e4-682b-4597-8a96-cd2fdeb11e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:50.000554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 146/296] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:50.020758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a002d-7267-4212-a1cb-58dff63f6d67 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.142514 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a002d-7267-4212-a1cb-58dff63f6d67 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.179926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.180829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.181120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.181400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.185019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d5f0-919b-4881-b1f3-916237704bf8 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.186121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d5f0-919b-4881-b1f3-916237704bf8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.187298 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5d5f0-919b-4881-b1f3-916237704bf8 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.189709 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 147/297] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:50.192771 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.194707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c293-90cf-41bb-bccc-6086d0ceec14 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.195445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c293-90cf-41bb-bccc-6086d0ceec14 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.195803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38c293-90cf-41bb-bccc-6086d0ceec14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.201342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:50.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:50.210606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:50.216490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 151/298] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:50.221505 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.222099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-6b38c293-90cf-41bb-bccc-6086d0ceec14 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:50.223284 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-6b38c293-90cf-41bb-bccc-6086d0ceec14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:50.306208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 148/299] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:50.310538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d0c1-a850-4b24-80d4-b5a03872b0db None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.310923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d0c1-a850-4b24-80d4-b5a03872b0db None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.311288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1d0c1-a850-4b24-80d4-b5a03872b0db None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.313315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 149/300] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:50.318796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.319086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.319371 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.336646 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:50.337909 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:50.350731 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.356742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840f297-1526-4264-9ad7-70410ceb1e3a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.393786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.394382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.394673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.394948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.395713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:50.396027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:50.397455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:50.411249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 152/301] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:50.411249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.420946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 150/302] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:50.426160 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.480266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.528994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188af43-b075-4d43-8c39-a76671c37fac None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.539223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667eda0d-3a8c-4a2b-b8c7-5eb290f82764 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.569582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.570395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.570687 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.570968 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.581983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:50.583032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.584009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.584297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.584607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] PATH_INFO: `/v3/users/ad529be7caf74f8f86b26408bd3d020e` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.585829 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] RBAC: Authorizing `identity:get_user(user_id=ad529be7caf74f8f86b26408bd3d020e)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:50.587295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:50.594112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/users/ad529be7caf74f8f86b26408bd3d020e => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:50.594340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:50.595327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:50.596256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:50.597024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.597115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:50.597115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.615133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.722440 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120062-8853-4b56-ac1c-4725281db381 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.725171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68404f1-2c76-49bf-9b5d-2c2012894731 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.744204 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.761931 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.762907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.763195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.763472 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.764770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:50.766250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:50.766374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.767233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.767523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.767801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] PATH_INFO: `/v3/system/users/ad529be7caf74f8f86b26408bd3d020e/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.769338 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ad529be7caf74f8f86b26408bd3d020e, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:50.770705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:50.772720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:50.773398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.780685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 154/306] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 20:24:50 2022] PUT /identity/v3/system/users/ad529be7caf74f8f86b26408bd3d020e/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:24:50.781475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:50.920269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:50.937922 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e41c0f-6e0f-4e1c-9ca7-7982b56f2bd8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.950215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f8ed7-25ae-4d15-b75e-619ecceaaf2f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:50.982525 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.983301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:50.984217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:50.984732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] PATH_INFO: `/v3/groups` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:50.985442 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:50.987022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:50.998256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:50.999664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.000490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.001063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.002086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:51.002653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:51.004465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:51.005233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 155/307] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 20:24:50 2022] POST /identity/v3/groups => generated 229 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:24:51.017280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 153/308] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:51.023400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:51.085298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:51.158350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b320c-b73c-4cbf-ace9-9aa8fbd0f30a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:51.201965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:51.203283 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.203731 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.204169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] PATH_INFO: `/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.221604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] RBAC: Authorizing `identity:get_user(user_id=ab9c4e75a8e94cd5b46c16b4c121da9d)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:51.223426 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f27d3a-e59c-4aa7-8ca7-42d9d373de46 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:51.225766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 156/309] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:51.231137 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:51.246592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7a22-dabd-46b0-91b8-6289cbdce26b None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.247163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7a22-dabd-46b0-91b8-6289cbdce26b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.247629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f7a22-dabd-46b0-91b8-6289cbdce26b None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.249672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 157/310] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:51.255641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547917-9684-40d0-a39a-be74a9026570 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.255982 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547917-9684-40d0-a39a-be74a9026570 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.256215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547917-9684-40d0-a39a-be74a9026570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.278239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-de547917-9684-40d0-a39a-be74a9026570 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:51.280661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-de547917-9684-40d0-a39a-be74a9026570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:51.361399 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 158/311] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:51.362874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62edf7-50b1-4f39-965f-f5e8ba205d95 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:51.385453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547917-9684-40d0-a39a-be74a9026570 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:51.401599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db844-2181-40fa-b212-23e234581c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:51.402277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db844-2181-40fa-b212-23e234581c23 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.402565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db844-2181-40fa-b212-23e234581c23 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.402853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844db844-2181-40fa-b212-23e234581c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.403693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db844-2181-40fa-b212-23e234581c23 None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, user.id=ab9c4e75a8e94cd5b46c16b4c121da9d, scope.system=all) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:51.404005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db844-2181-40fa-b212-23e234581c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:51.405793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844db844-2181-40fa-b212-23e234581c23 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:51.411689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 154/312] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/role_assignments?user.id=ab9c4e75a8e94cd5b46c16b4c121da9d&scope.system=all&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:51.420843 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:51.507455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547917-9684-40d0-a39a-be74a9026570 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:51.548969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:51.549806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.550156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.550442 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.563512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:51.572021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:51.573794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:51.575032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 159/313] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:24:51.583169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db844-2181-40fa-b212-23e234581c23 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:51.604925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:51.706327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844db844-2181-40fa-b212-23e234581c23 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:51.742840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:51.743540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.743827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.744104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.744906 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:51.745217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:51.746622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:51.757156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 155/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:51.762752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:51.884890 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235fc69-47d8-4f9b-8178-f7267c936edb None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:51.888314 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:51.931366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:51.932762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:51.933278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:51.933793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] PATH_INFO: `/v3/users/f0cd4643c31446a0990c4870361f7151` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:51.935243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] RBAC: Authorizing `identity:get_user(user_id=f0cd4643c31446a0990c4870361f7151)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:51.937686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:51.940510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 160/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/users/f0cd4643c31446a0990c4870361f7151 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:51.945903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:52.028018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:52.069000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f5c68-ccfe-4cb5-be06-d67f8143b3aa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:52.107917 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:52.110163 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:52.110908 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:52.111608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:52.193036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:52.194986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:52.275664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:52.322144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 156/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:24:52.327751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:52.450745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf372f-dbcf-40cd-b29e-801da768a7a6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:52.490365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:52.491251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:52.491546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:52.491835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:52.493088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:52.494554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b82f0-bff4-4b32-8ecb-7e31909482b6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:52.496304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 161/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:52 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:52.502996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:52.613640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fb5aa-b655-419e-bb34-0914610d4385 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:52.652684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:52.653360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:52.653647 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:52.653930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:52.654780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, scope.project.id=5c8bf372028942759324fafea37a5438, user.id=f0cd4643c31446a0990c4870361f7151) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:52.655095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:52.657246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d3384-daf2-4ea5-a1f7-981c8315ecb0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:52.664048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 157/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:24:52 2022] GET /identity/v3/role_assignments?user.id=f0cd4643c31446a0990c4870361f7151&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:53.214342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b14d-9eec-431a-be73-7d6c51892714 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.214919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b14d-9eec-431a-be73-7d6c51892714 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.214919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b14d-9eec-431a-be73-7d6c51892714 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.216462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 162/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:53.222445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e50a-cee6-458b-869e-9b31913868f3 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.222745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e50a-cee6-458b-869e-9b31913868f3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.223034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3e50a-cee6-458b-869e-9b31913868f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.240841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a0b3e50a-cee6-458b-869e-9b31913868f3 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:53.242036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a0b3e50a-cee6-458b-869e-9b31913868f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:53.274107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:53.324232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 158/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:53.348500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b14d-9eec-431a-be73-7d6c51892714 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:53.379433 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:53.460393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b14d-9eec-431a-be73-7d6c51892714 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:53.498001 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:53.498723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.499020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.499824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.512211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:53.513796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3680b9-4e50-47e6-a2e4-1d5c1a7e4d40 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.514100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3680b9-4e50-47e6-a2e4-1d5c1a7e4d40 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.514388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3680b9-4e50-47e6-a2e4-1d5c1a7e4d40 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.516100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 159/321] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:53.520469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:53.521495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:53.522466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:53.523188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.523188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.523188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.523411 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 163/322] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:53.529367 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:53.538985 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:53.540253 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:53.622998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 160/323] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:24:53.636918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:53.646243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:53.658137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdb1ab-4cbe-4d9d-acd0-3d631b7db395 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:53.703147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:53.704156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.704471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.704756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.705522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:53.705834 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:53.707234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:53.717342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 164/324] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:53.724125 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:53.758237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:53.768044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab8d2-5c52-4715-8f52-45ad6a0ed2de None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:53.805580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:53.806380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.806663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.806938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.808103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:53.809592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be883d-16e7-4b52-829a-e5c32d7f7218 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:53.811225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 161/325] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:24:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:53.814173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc55f-292e-40aa-8ccd-b3eff4f4a061 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.814467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc55f-292e-40aa-8ccd-b3eff4f4a061 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.814869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfc55f-292e-40aa-8ccd-b3eff4f4a061 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.816185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 162/326] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:53.817548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc55f-292e-40aa-8ccd-b3eff4f4a061 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:53.843798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66081df4-f6c1-424e-a36c-41513d308e65 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:53.880347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:53.881278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.881562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.881838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] PATH_INFO: `/v3/users/ad529be7caf74f8f86b26408bd3d020e` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.883067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] RBAC: Authorizing `identity:get_user(user_id=ad529be7caf74f8f86b26408bd3d020e)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:53.884560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac4739-5886-4823-8da8-a21d11b537c5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:53.886324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 165/327] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:53 2022] GET /identity/v3/users/ad529be7caf74f8f86b26408bd3d020e => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:24:53.887494 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.887785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.888060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.904295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:53.906143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:53.916801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:53.941462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfc55f-292e-40aa-8ccd-b3eff4f4a061 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:53.977500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:53.978041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:53.978326 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:53.978606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] PATH_INFO: `/v3/groups` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:53.979310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:53.980170 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 166/328] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:53.980812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:53.981689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:54.006520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 163/329] 10.222.0.13 () {60 vars in 1134 bytes} [Thu May 19 20:24:53 2022] POST /identity/v3/groups => generated 226 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:24:54.007456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:54.050379 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:54.102870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a6edb-d1a2-4807-a9b1-405e37af3f71 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:54.144347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:54.145113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:54.145405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:54.145684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:54.146556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] RBAC: Adding query filter params (role.id=5f236360f48f4915aae4c33e16ec68e0, user.id=ad529be7caf74f8f86b26408bd3d020e, scope.system=all) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:54.146875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:54.148352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:54.152567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6320fa6-c300-483a-94ea-7423db416f4f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:54.167211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 167/330] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:24:53 2022] GET /identity/v3/role_assignments?user.id=ad529be7caf74f8f86b26408bd3d020e&scope.system=all&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:54.197186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:54.198042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:54.198330 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:54.198620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:54.215504 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:54.224008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:54.225315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:54.226231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 164/331] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:54.231747 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:54.251391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:54.385267 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c63ec-d01f-48e2-b711-be99ef1bb96b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:54.425107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:54.425799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:54.426091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:54.426374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:54.427149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:54.427463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:54.428898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:54.439057 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 168/332] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:54.445171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:54.550622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:54.573891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba232d57-4cc9-4799-9422-a2b6d46d512a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:54.619459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:54.621065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:54.621636 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:54.622142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] PATH_INFO: `/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:54.623722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] RBAC: Authorizing `identity:get_user(user_id=ab9c4e75a8e94cd5b46c16b4c121da9d)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:54.626014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7268f9c-b574-43ab-bf98-8222f838e5ee None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:54.628250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 165/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:24:54 2022] GET /identity/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:54.634215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:54.725916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:54.777912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b324008-ac9b-407a-9cda-a08da75071f8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:54.823683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:54.824665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:54.824956 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:54.825239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] PATH_INFO: `/v3/system/users/ab9c4e75a8e94cd5b46c16b4c121da9d/roles/f589e1251c7e4ffb903f695560d349a8` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:54.827740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ab9c4e75a8e94cd5b46c16b4c121da9d, role_id=f589e1251c7e4ffb903f695560d349a8)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:54.829336 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe1a213-3d17-42f5-a664-fc26bf1c4ce2 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:54.839405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 169/334] 10.222.0.13 () {58 vars in 1334 bytes} [Thu May 19 20:24:54 2022] PUT /identity/v3/system/users/ab9c4e75a8e94cd5b46c16b4c121da9d/roles/f589e1251c7e4ffb903f695560d349a8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:24:56.963577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e017a5-4b08-4b11-892d-f09444fd30eb None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:56.964121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e017a5-4b08-4b11-892d-f09444fd30eb None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:56.964121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e017a5-4b08-4b11-892d-f09444fd30eb None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:56.965849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 166/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:56.971577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90401610-849a-4cff-a466-1e27fc60935c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:56.972218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90401610-849a-4cff-a466-1e27fc60935c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:56.972218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90401610-849a-4cff-a466-1e27fc60935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:56.996342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-90401610-849a-4cff-a466-1e27fc60935c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:56.997696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-90401610-849a-4cff-a466-1e27fc60935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:57.102029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 170/336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:57.146606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e017a5-4b08-4b11-892d-f09444fd30eb None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:57.231027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6808-68c1-4878-8250-8f521a48d6f4 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.231438 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6808-68c1-4878-8250-8f521a48d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.231705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c6808-68c1-4878-8250-8f521a48d6f4 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.237731 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 171/337] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:57.248121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795371-2c0f-475a-a4fb-bde7a5229baa None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.248121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795371-2c0f-475a-a4fb-bde7a5229baa None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.248121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795371-2c0f-475a-a4fb-bde7a5229baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.289534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-4a795371-2c0f-475a-a4fb-bde7a5229baa None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:57.291142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-4a795371-2c0f-475a-a4fb-bde7a5229baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:57.309281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e017a5-4b08-4b11-892d-f09444fd30eb None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:57.381669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:57.382427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.382717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.383001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.406098 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-41b72a11-d449-41e2-81ec-a3b71d267e5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:57.424767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:57.425872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:24:57.426776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:57.427490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:57.427571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 167/338] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:24:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:57.427644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d322b3-7ee9-498c-9656-16f89a2cf88f None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.428591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d322b3-7ee9-498c-9656-16f89a2cf88f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.435710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d322b3-7ee9-498c-9656-16f89a2cf88f None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.437979 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 168/339] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:57.442001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d322b3-7ee9-498c-9656-16f89a2cf88f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:57.444778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 172/340] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:57.448785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.449139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.449435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.475096 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:57.491365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:57.493611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:57.653144 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 173/341] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:57.654169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:57.708045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d322b3-7ee9-498c-9656-16f89a2cf88f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:57.803719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:57.804955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.805450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.805937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.807437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:57.808030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:57.810734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:57.845045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 169/342] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:24:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:57.846470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:57.911774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9feb1f-ae63-4217-9f3d-3616b467b74e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:57.945836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:57.974532 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:57.975352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:57.975639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:57.975917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:57.997299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:58.014158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:58.015368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:58.016460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 174/343] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:58.018193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.111058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff6666-585a-465f-a143-e638d6857d9b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.153493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d91ab-9a15-4c32-89c7-22b86428c595 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.192582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.193272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.193562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.193841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.207158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.207943 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.208238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.208558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.234104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:58.242859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:58.244141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:24:58.247843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:24:58.250978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 170/344] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:58.251988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.260821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.282284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:58.283775 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa6204-9ae8-4e2e-b943-f2379ac69b26 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:58.414009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed4efc-0410-4226-98e1-3cb2bd4424b5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.429583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 175/345] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:58.431112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11de8a-6843-4760-8ba7-44ced9fbb9b5 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.431409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11de8a-6843-4760-8ba7-44ced9fbb9b5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.431688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11de8a-6843-4760-8ba7-44ced9fbb9b5 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.433112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 176/346] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:58.433820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11de8a-6843-4760-8ba7-44ced9fbb9b5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.458643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.471794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.471794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.472289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.472675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.473567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:58.473960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:58.476517 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:58.491302 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 171/347] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:58.492257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.555526 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.589404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11de8a-6843-4760-8ba7-44ced9fbb9b5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.631924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29ca79-c2bb-4130-a1a4-a79930e820ed None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.644379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.645169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.645467 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.645755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.646578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:58.646880 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:58.649761 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8acda2-f857-4b64-9f55-ea602de76c22 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:58.663136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 177/348] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:58.664997 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.665508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.665944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.687948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.687948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.688289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.688613 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.690364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:58.691909 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:58.694406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 172/349] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:58.694680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.709137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:58.713488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:58.776239 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.822960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 178/350] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:58.824522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:58.829144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be729ac3-8d1d-4ce2-85e7-f23dae5db15e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:58.887770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:58.888786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:58.889277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:58.890218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:58.944823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:58.990849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:58.992318 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:58.993330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a257f5-40aa-4285-a4e4-4bb6e9e9a368 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.045987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.047302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.047598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.047889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.082989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.133302 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 173/351] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:59.134184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.146879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:59.148769 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:59.238586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.273221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b41f0-30a6-4e1a-908c-1a2fe43d3592 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.285135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 179/352] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:24:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:24:59.285910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.325118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.326024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.326315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.326598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.334637 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:59.337203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5bfca-8072-4faa-bb6e-6d6b944b9805 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:59.339486 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 174/353] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:59.340998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09811-7414-492c-be63-2be5cdc71f69 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.341468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09811-7414-492c-be63-2be5cdc71f69 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.341918 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a09811-7414-492c-be63-2be5cdc71f69 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.344500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 175/354] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:59.346276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5208-92a6-402f-893b-340710dd3537 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.346756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5208-92a6-402f-893b-340710dd3537 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.347541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca5208-92a6-402f-893b-340710dd3537 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.349771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 176/355] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:24:59.351115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca5208-92a6-402f-893b-340710dd3537 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.388231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.424856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee0c65-dc04-457b-90f1-4712aae2406c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.469374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.470209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.470504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.471724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.483682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:24:59.491714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:24:59.492909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 180/356] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:24:59.494396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.495984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca5208-92a6-402f-893b-340710dd3537 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.541801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.542316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.542466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.542622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.542871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.544082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:59.545587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:59.552457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 177/357] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:59.552457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.636169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a553-cbac-435b-9627-773dc499e435 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.670051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.679907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155668f-bbb1-4cfc-ad87-48a82722dc6b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.684440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.684440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.684440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.684792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.687141 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:59.688682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1196def-5621-4d17-b9c8-6d318e7ce05e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:59.690744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 181/358] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:59.691981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.692279 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.692618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.712162 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:59.713406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:59.725479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:24:59.726497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.726783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.727065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.728073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=c27ff1b3891041938db6c68e900e9136) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:24:59.728390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:24:59.729917 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a27c46-cad5-46bb-97dd-ca13960a0e06 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:24:59.737960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 178/359] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:24:59.739288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:24:59.739579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:24:59.739856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:24:59.759773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:24:59.761122 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:24:59.808370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 182/360] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:59.809844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.850016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 179/361] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:24:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:24:59.851352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:24:59.854636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.953932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489bc4-10d0-43ff-9396-e9ba2a03f18b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:24:59.990721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:24:59.998863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.000032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.000427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.000733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.001496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] RBAC: Adding query filter params (name=reader) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:00.001920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.003196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.006941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d011dc7-2115-44e6-9818-ee1d32371e7a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.014119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 183/362] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.015553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.059593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.060675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.061069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.061973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.063337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.065179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.067941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 180/363] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:24:59 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.068779 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.145286 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:00.157570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0218d-b774-466b-a278-7b3790e23c93 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.205432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.206619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.206911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.207196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.207851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970525e-e365-43e3-a005-0f02be44a866 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.208471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.210136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.213114 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 184/364] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.213941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.256647 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.257478 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.257766 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.258128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.274081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:00.284725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:00.286063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:00.287260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:00.290262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 181/365] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:00.296468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.332986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:00.388947 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd46b56-8d04-4db3-9f73-43ed52959a07 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.436363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.437278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.437569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.437944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.454604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:00.458795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7ca0-dff1-455c-bf4a-362b0ddc5045 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:00.462513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:00.463744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:00.464748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 185/366] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:00.465529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.490856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:00.520621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.521522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.521816 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.522346 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] PATH_INFO: `/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.523340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] RBAC: Authorizing `identity:get_user(user_id=ab9c4e75a8e94cd5b46c16b4c121da9d)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.527144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.529058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 182/367] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/users/ab9c4e75a8e94cd5b46c16b4c121da9d => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.529763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.597518 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5cf06-d47a-4436-af65-289848802d4f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.638325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.639000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.639290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.639632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.640524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=c27ff1b3891041938db6c68e900e9136) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:00.640844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.642341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.646480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:00.657469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 186/368] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.658398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.681188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa76f5-16ea-4349-b9ec-e67b4864f16b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.735215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.736998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.737481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.738233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.739553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=5c8bf372028942759324fafea37a5438) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:00.740049 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.742599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.759330 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 183/369] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.759330 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:00.809740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:00.850544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d179422-27c9-4c3c-b68c-999c0c9bb1d9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:00.945899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:00.946608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:00.946897 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:00.947180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:00.948031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:00.948412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:00.949829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:00.972234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 187/370] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:00.972234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:01.013990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e2a70-2cc1-4ec6-b74a-fdde2f6bcb51 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:01.094277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:01.094980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:01.095272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:01.095553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:01.096325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:01.096680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:01.098094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:01.106375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe91839-22b2-40d3-92ef-94b9ebcff516 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:01.107129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 184/371] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:01.107984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:01.144787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:01.151798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:01.152526 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:01.152819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:01.153102 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:01.153950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] RBAC: Adding query filter params (role.id=f589e1251c7e4ffb903f695560d349a8, user.id=ab9c4e75a8e94cd5b46c16b4c121da9d, scope.system=all) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:01.154269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:01.157024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:01.170600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 188/372] 10.222.0.13 () {56 vars in 1308 bytes} [Thu May 19 20:25:00 2022] GET /identity/v3/role_assignments?user.id=ab9c4e75a8e94cd5b46c16b4c121da9d&scope.system=all&role.id=f589e1251c7e4ffb903f695560d349a8 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:01.176731 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:01.186268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:01.356727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d31e47-a435-465b-8991-41b65b028d46 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:01.431467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:01.432713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:01.433257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:01.433788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:01.488870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf182b1-62b4-411e-9698-879e2f7ddcc8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:01.569128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:01.570192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:01.570716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:01.571235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:01.579902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:01.587640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:01.609488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:01.714542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:01.716017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:01.720872 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:01.738097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 185/373] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:01.745656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:01.887765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee36-40de-494c-ac2c-836644d3099f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:01.908454 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:01.946698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:01.947970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:01.948515 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:01.949048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:01.950519 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:01.953814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:01.963690 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 186/374] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:01 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:01.965409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.015841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 189/375] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 836 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:02.018914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.177450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33746e66-edb7-4ea2-8e61-c2fbb508dd4a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.230386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b6275-e6de-4352-93b3-9d51587e2d0a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.234555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:02.255177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.260179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.260815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.261332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.264003 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:02.266895 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:02.270869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 187/376] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:01 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:02.282983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.285396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.286583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.287093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.287615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] PATH_INFO: `/v3/projects/b4631908c3a341389e87c50c9ca6a7c4` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.289248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] RBAC: Authorizing `identity:get_project(project_id=b4631908c3a341389e87c50c9ca6a7c4)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:02.290976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:02.298402 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 190/377] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/projects/b4631908c3a341389e87c50c9ca6a7c4 => generated 344 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:02.300185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.406712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:02.416095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49885114-6cab-4c03-8cb7-aeb83652fced None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.457077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8a9f9-2406-4812-836a-1d61694ef51a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.458661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.459983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.460277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.460605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] PATH_INFO: `/v3/groups/bcc0b5ec7f2b42818ea6436565c76783` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.467459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0b5ec7f2b42818ea6436565c76783)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:02.469678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eac192-05ef-4e8c-ad6c-776dc80b3c30 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:02.473621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 191/378] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/groups/bcc0b5ec7f2b42818ea6436565c76783 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:02.474532 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8805-4eda-4aab-90cb-a290526dd7b5 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.474825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8805-4eda-4aab-90cb-a290526dd7b5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.475107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf8805-4eda-4aab-90cb-a290526dd7b5 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.476789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 192/379] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:02.481593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8805-4eda-4aab-90cb-a290526dd7b5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.512016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.513408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.513918 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.515261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.516972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=5c8bf372028942759324fafea37a5438) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:02.517638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:02.520362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fad09-2415-489a-8963-a209401868e3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:02.533522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 188/380] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:02.534160 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.534460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.534744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.534985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:02.575955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:02.579778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:02.678970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 189/381] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:02.690711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf8805-4eda-4aab-90cb-a290526dd7b5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.706397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:02.763562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e06cf-998e-4535-8095-673237c31616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.764166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e06cf-998e-4535-8095-673237c31616 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.764495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e06cf-998e-4535-8095-673237c31616 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.764495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e06cf-998e-4535-8095-673237c31616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.765669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e06cf-998e-4535-8095-673237c31616 None admin] RBAC: Adding query filter params (group.id=bcc0b5ec7f2b42818ea6436565c76783, role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=b4631908c3a341389e87c50c9ca6a7c4) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:02.765984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e06cf-998e-4535-8095-673237c31616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:02.767462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e06cf-998e-4535-8095-673237c31616 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:02.784257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 193/382] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/role_assignments?group.id=bcc0b5ec7f2b42818ea6436565c76783&scope.project.id=b4631908c3a341389e87c50c9ca6a7c4&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 260 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:02.794923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:02.877117 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db31486-828a-462a-8d51-d0ceba18777c None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:02.930671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:02.931506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:02.931792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:02.932073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:02.954208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:02.962714 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:02.963811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:02.964777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:02.968597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 190/383] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:02.970542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e06cf-998e-4535-8095-673237c31616 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:03.134561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:03.138691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e06cf-998e-4535-8095-673237c31616 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:03.183452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:03.184140 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.184453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.184737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.185502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:03.185813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:03.188137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:03.197835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 194/384] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:03.204821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:03.338282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:03.341960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a206f98-1968-4529-aa6f-6517eb35f068 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:03.391794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:03.392700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.392990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.393271 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.394506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:03.395977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:03.397832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 191/385] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:03 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:03.403094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:03.550410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f1c1-33c7-44cd-887c-f317aeec23cf None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:03.603647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:03.604615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.604908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.605189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.606412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:03.607858 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e0021-d3d3-4a65-94b3-fcba055b764b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:03.610438 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 195/386] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:03 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:03.616053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:03.745341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190f3-95bd-4e43-bc11-9c217ec461f2 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.745539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190f3-95bd-4e43-bc11-9c217ec461f2 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.745820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055190f3-95bd-4e43-bc11-9c217ec461f2 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.747350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 196/387] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:03.754521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8925ad-553d-44ce-870f-a6d48adb83dc None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.754823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8925ad-553d-44ce-870f-a6d48adb83dc None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.755218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8925ad-553d-44ce-870f-a6d48adb83dc None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.756783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 197/388] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:03.757889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.758182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.758466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.761683 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4cff-611c-4da6-bcc1-dceda43b2c6c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:03.782767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:03.784049 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:03.805562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:03.806462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.806746 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.807143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.809893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, group_id=7517cec076ce4cd8b1ab5b03b306144a, role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:03.812454 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f9a9c-37e3-4407-9267-94b59c7ae42f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:03.820163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:03.832259 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 192/389] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:25:03 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:03.839618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:03.839618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:03.839618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:03.883332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:03.884793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:03.925324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 198/390] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:03.935567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:03.981439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.016644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 193/391] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:04.074097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.122710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216c273-bfbc-4576-bf6f-f87ea0686742 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.162816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.164086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.164640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.165125 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.177326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:04.185757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:04.187352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:04.188302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:04.191522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 199/392] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:04.199159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2f7018-95d1-4698-8858-95bf808df5f5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.200119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.225307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:04.239699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.240548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.240856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.241364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.254809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:04.263144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:04.264184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:04.265937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:04.272058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 194/393] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:04.272058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.293347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:04.419970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cdfcf-d2a2-4d90-ab09-08ed22b359f9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.424009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d39c72-d1a3-4b54-833f-d5ba91ea46fa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.483074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.483378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.483378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.483378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.483378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.483751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:04.483751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:04.483751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:04.484802 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.485252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.485676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.486593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:04.487053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:04.488614 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:04.492237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 195/394] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:04.498679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 200/395] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:04.500620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.505479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.597321 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:04.622344 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e4cd0c-f6ef-48b2-9eef-1f5075ebf94c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.670445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.672898 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.673607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.674310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.677022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:04.679154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:04.681546 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 196/396] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:04.688211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.720528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:04.766546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ad2e9-938e-4ed1-844e-4f616f2988e3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.835209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:04.836093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:04.836382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:04.836688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:04.838641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:04.840106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:04.842118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 201/397] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:04.852975 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:04.929646 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7c32b-c50d-456f-b307-139d3719ee23 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:04.998062 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.000470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.001013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.001964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.003477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:05.005704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:05.008015 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 197/398] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:05.013582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:05.039214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:05.050809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e10c8-0017-47ff-a68b-a2caaa2f2159 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:05.146318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.147592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.148110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.148629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.151197 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:05.153891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:05.158334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 202/399] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:04 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:05.164382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:05.170827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e362962-e126-4396-8951-8d0dcb7aa92b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:05.215968 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:05.240153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.241475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.241978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.242502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.244865 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, group_id=7517cec076ce4cd8b1ab5b03b306144a, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:05.246601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e163483-a219-429f-a22d-f1ef78df41da None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:05.274618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 198/400] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:25:05 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:05.276096 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffebf9-fdb5-47fe-85b8-af84b05665a3 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.276394 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffebf9-fdb5-47fe-85b8-af84b05665a3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.276755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffebf9-fdb5-47fe-85b8-af84b05665a3 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.284640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 199/401] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:05.296684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.296684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.297440 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.328437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff6f2f-272c-4039-8eb3-eeed4a108ec3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:05.330338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:05.331639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:05.385546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.386457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.386832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.387316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/5f236360f48f4915aae4c33e16ec68e0` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.394003 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] RBAC: Authorizing `identity:create_grant(project_id=c27ff1b3891041938db6c68e900e9136, group_id=7517cec076ce4cd8b1ab5b03b306144a, role_id=5f236360f48f4915aae4c33e16ec68e0)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:05.395545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:05.426983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 203/402] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:25:05 2022] PUT /identity/v3/projects/c27ff1b3891041938db6c68e900e9136/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/5f236360f48f4915aae4c33e16ec68e0 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:05.513961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 200/403] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:05.563259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:05.689748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:05.717905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0f61a-fdef-4b8c-a909-d0b43644944b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:05.766860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.768053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.768630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.769135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.781944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-debc7eb5-54a3-452e-bacb-75e37e9c2224 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:05.790427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:05.791464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:05.792431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:05.795469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 204/404] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:05.800842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:05.811183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:05.820492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf5da-928b-4e20-b469-b2a658656aa8 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.820838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf5da-928b-4e20-b469-b2a658656aa8 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.821068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf5da-928b-4e20-b469-b2a658656aa8 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.822722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 205/405] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:05.828480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.828765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.829052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.877461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:05.878676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:05.917701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb428f0b-429d-455b-9218-3c91827fc8a1 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:05.955843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:05.956471 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:05.956896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:05.957246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:05.958145 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:05.958527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:05.960162 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:05.970908 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 201/406] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:05.976320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.072848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:06.094018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2a6d6-0af3-4b6a-a82e-90d86c66c4bc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:06.109663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 206/407] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:06.138575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:06.140182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.140743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.141172 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] PATH_INFO: `/v3/groups/bcc0b5ec7f2b42818ea6436565c76783` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.142586 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0b5ec7f2b42818ea6436565c76783)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:06.144010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.158843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:06.163258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 202/408] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:05 2022] GET /identity/v3/groups/bcc0b5ec7f2b42818ea6436565c76783 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:06.168099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.303796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb938ba9-f420-4b01-8af1-5cbca98c2d54 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:06.344089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:06.345013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.345788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.346087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] PATH_INFO: `/v3/projects/b4631908c3a341389e87c50c9ca6a7c4` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.347396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] RBAC: Authorizing `identity:get_project(project_id=b4631908c3a341389e87c50c9ca6a7c4)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:06.348964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:06.351270 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 203/409] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:06 2022] GET /identity/v3/projects/b4631908c3a341389e87c50c9ca6a7c4 => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:06.356948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.364662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:06.368798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:06.478506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512e6f4-01b5-46a6-854c-d55dbd52e2e8 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:06.500485 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e478294-162d-4dfa-bd9b-8c2b52b8bc12 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:06.540707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:06.542050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.542349 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.542633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] PATH_INFO: `/v3/projects/b4631908c3a341389e87c50c9ca6a7c4/groups/bcc0b5ec7f2b42818ea6436565c76783/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.544734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=b4631908c3a341389e87c50c9ca6a7c4, group_id=bcc0b5ec7f2b42818ea6436565c76783, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:06.546923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5e875-b4ec-4879-971d-04b7b136efa8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:06.570558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 204/410] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:25:06 2022] PUT /identity/v3/projects/b4631908c3a341389e87c50c9ca6a7c4/groups/bcc0b5ec7f2b42818ea6436565c76783/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:06.583469 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559829db-c589-4b6b-b3e8-6384143afe1a None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.584093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559829db-c589-4b6b-b3e8-6384143afe1a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.584616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559829db-c589-4b6b-b3e8-6384143afe1a None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.586694 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 205/411] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:06.593242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.593525 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:06.594039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.594166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.594373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.594655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.594924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.613978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:06.615557 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:06.622576 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:06.637682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:06.639016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:06.640699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 207/412] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 495 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:06.649502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.720457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:06.740294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 206/413] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:06.814860 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:06.833073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af578dba-5b07-49f5-93bf-934825e69606 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:06.865419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:06.873710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:06.874728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:06.875206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:06.875656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:06.876621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:06.877096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:06.878671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:06.888557 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 208/414] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:06.894508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:07.016512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186b363-9f59-42f6-9a22-4c52652c3f21 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.060580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.061794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.062267 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.062720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.065096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:07.066734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:07.070480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 209/415] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:06 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:07.074609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:07.084625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2356d7d7-7ad6-4732-8d09-fbb99bb7d8e8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.148004 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:07.154635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.156130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.156436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.156724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.189552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:07.195967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65387e-65a1-47cf-82ab-0cd0cb4d1a63 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:07.205535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:07.206811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:07.208067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:07.209821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 207/416] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:07.218869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:07.279553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.281120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.281514 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.281899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.283251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:07.284888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:07.287300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 210/417] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:07 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:07.295239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:07.320790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:07.411904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5074206-a0af-447a-af40-f688ac851ab9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.477589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.478302 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.478593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.478879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.479273 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c20946-7886-48ff-985e-f58cf26bec1f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.479739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:07.479982 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:07.481460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:07.509830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 208/418] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:07.515829 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:07.536284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:07.558989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.559891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.560177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.560477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.566847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8bf372028942759324fafea37a5438, group_id=7517cec076ce4cd8b1ab5b03b306144a, role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:07.568298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:07.592428 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 211/419] 10.222.0.13 () {58 vars in 1442 bytes} [Thu May 19 20:25:07 2022] PUT /identity/v3/projects/5c8bf372028942759324fafea37a5438/groups/7517cec076ce4cd8b1ab5b03b306144a/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:07.665016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fb65c-2862-442e-b5dc-aa82b8195691 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:07.691005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:07.714357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:07.715053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:07.715342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:07.715628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:07.799497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:07.801046 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbceee-0244-4099-8e3c-54b2accbb9c1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:07.949045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 209/420] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:25:07.955488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.111563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4257f-1d60-449a-b5a4-ebc0185a02b8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.142892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fcd5-42d4-48aa-8013-f91f8257e370 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.143226 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fcd5-42d4-48aa-8013-f91f8257e370 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.143454 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fcd5-42d4-48aa-8013-f91f8257e370 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.145267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 210/421] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:08.151478 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5ee4-4127-4e03-ad8d-a7a2660763fd None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.151766 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5ee4-4127-4e03-ad8d-a7a2660763fd None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.152054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5ee4-4127-4e03-ad8d-a7a2660763fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.163994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.165466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.165958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.166391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.167793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:08.169504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:08.172547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 212/422] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:07 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:08.177826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.180836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-008c5ee4-4127-4e03-ad8d-a7a2660763fd None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:08.182005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-008c5ee4-4127-4e03-ad8d-a7a2660763fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:08.201522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:08.308805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:08.333092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 211/423] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:08.340427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2e95-77be-4be8-a783-7369bab4a64d None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.340885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2e95-77be-4be8-a783-7369bab4a64d None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.341177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc2e95-77be-4be8-a783-7369bab4a64d None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.343383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 212/424] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:08.353814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.354094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.354377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.376939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:08.378734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:08.380293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0ee67-ed5b-4b01-836c-66b9918dfedc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.419661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.421416 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.421710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.421994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.423260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:08.424774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:08.432569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 213/425] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:08.435841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.471764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 213/426] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:08.471764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.569178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:08.589868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d53c89-95cd-45bf-bd98-223a2810fd96 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.598719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c41cd8-a322-461f-a22f-79a9c1e098d7 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.629145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:08.645980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.646795 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.647079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.647362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.659608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.660621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.660924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.661210 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.662075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=c27ff1b3891041938db6c68e900e9136) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:08.662400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:08.664257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:08.664709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:08.675143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:08.676254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:08.677264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:08.680368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 214/428] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:08.680544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 214/428] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:08.681427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.691610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.884992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd928-0feb-48de-b36f-58cae7172a1d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.929315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.930009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.930362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.930644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:08.931689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:08.932018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:08.934177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:08.948311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 215/429] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:08.955031 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296f9d0-7522-4676-8843-369ae54942a0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:08.957736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:08.996563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:08.997439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:08.997733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:08.998014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] PATH_INFO: `/v3/roles/member` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.008749 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.015928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:09.022045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:09.025193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:09.026231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:09.027254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 215/430] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:09.037498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.096391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea776d-e7aa-4f01-a408-592b92ccd4f5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.141575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.145752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.146586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.147121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.165255 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c8b8e-2556-4228-95df-2cfe4033099e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.206626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.207315 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.207610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.207900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.208707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:09.209032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.210656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.220743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 216/431] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:09.226021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.229539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.231007 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.282842 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.310424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.351462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 216/432] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:09.353051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.360082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebc355-e7a2-40e1-bd4d-2fca3d420cb8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.397935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.398633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.398923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.399202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.468078 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc856e-e794-4cd8-8d0f-b9b973b14ed7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.475375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.476909 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627343cd-2de5-4f88-817a-d8734cde4a4d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.514957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.515854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.516139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.516444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] PATH_INFO: `/v3/projects/c27ff1b3891041938db6c68e900e9136` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.517718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] RBAC: Authorizing `identity:get_project(project_id=c27ff1b3891041938db6c68e900e9136)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.519165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ee9a3-1163-4189-a57e-1e4cea1432a5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.525523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 217/433] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/projects/c27ff1b3891041938db6c68e900e9136 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:09.525523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06acd-929a-4d3b-b155-c39bc77256de None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.525523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06acd-929a-4d3b-b155-c39bc77256de None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.525523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a06acd-929a-4d3b-b155-c39bc77256de None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.525523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 218/434] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:09.526502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a06acd-929a-4d3b-b155-c39bc77256de None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.541920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.551831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.605036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 217/435] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:09.606362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.606660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.606944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.627467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:09.628702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:09.662473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a06acd-929a-4d3b-b155-c39bc77256de None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.718198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 218/436] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:09.718198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.721884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.723076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.723429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.723777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.725198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.726864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.729042 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 219/437] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:09.735174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.824607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.828940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:09.842269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4aa348-d6c8-434e-9cc3-e2f754965fda None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.852709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02619-232a-49d5-abd7-7e888dd19181 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:09.889060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.890016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.890306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.890584 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.891546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:09.891762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.892150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:09.892466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:09.892753 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:09.893255 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.893601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=c27ff1b3891041938db6c68e900e9136) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:09.893918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:09.895358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:09.899445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 219/438] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:09.900175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:09.912820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 220/439] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=c27ff1b3891041938db6c68e900e9136&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:09.912820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.041089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea36c6-d543-4254-8d53-501c0d206ba2 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.044163 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b5de24-111a-49a1-9a7b-9220a8285ecb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.086027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.087272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.087895 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.088353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.091427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.092627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.094203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.094639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.096289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.098022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.100043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 221/440] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:10.106455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.106922 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:10.116155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:10.117742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:10.119040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:10.121926 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 220/441] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:10.128184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.138018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.164093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.269340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8b28-2c52-4000-8928-26ff7843913d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.274922 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58389278-72b9-480b-8203-d8f9f7d2db42 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.312972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.313652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.314205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.314474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.315332 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=5f236360f48f4915aae4c33e16ec68e0, scope.project.id=5c8bf372028942759324fafea37a5438) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:10.315773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.317297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e7477-6340-46a9-a165-36a710c3aee7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.324071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 222/442] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=5f236360f48f4915aae4c33e16ec68e0 => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:10.325661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f75fa-b325-41ab-9453-51cce76fa684 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.325958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f75fa-b325-41ab-9453-51cce76fa684 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.326244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f75fa-b325-41ab-9453-51cce76fa684 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.326486 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.327738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 223/443] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:10.328055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.328349 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.328667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.329889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:10.329889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.331261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.334395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.334988 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.335482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.342634 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 221/444] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:10.347704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.360237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:10.361522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:10.458291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 224/445] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:10.468836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.483583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.493211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.495272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b8bc7-6e3a-4217-95e3-5bba978f9a22 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.551993 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.552880 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.553176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.553460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.626599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b4566-f340-40f9-8026-a90b39f46e41 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.654985 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.658919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.681942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.683563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.683881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.684170 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.699565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:10.714108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:10.715850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:10.717026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 225/446] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:10.722194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.774635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.798500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 222/447] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:25:10.804350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.805734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:10.853302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056819e-88c9-41a8-b77c-587bac22d60a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.892415 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.893114 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.893404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.893686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.894440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:10.894756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.896146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.904309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 226/448] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:10.910510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:10.933150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4abe3d-e36f-4338-9b7a-0313b2d893dc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:10.978087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:10.978880 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:10.979177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:10.979462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] PATH_INFO: `/v3/projects/b4631908c3a341389e87c50c9ca6a7c4` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:10.980704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] RBAC: Authorizing `identity:get_project(project_id=b4631908c3a341389e87c50c9ca6a7c4)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:10.982164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:10.984447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 223/449] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/projects/b4631908c3a341389e87c50c9ca6a7c4 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:10.989890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:11.018243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae361ea-7867-49ba-903e-d26d776dc5ef None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.043363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:11.045340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:11.054265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.055055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.055295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.055580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.098230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e775ec-9746-4f9e-a054-b60e3a6d37fc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.130292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.131790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.135971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.136881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.137171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.137452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] PATH_INFO: `/v3/groups/bcc0b5ec7f2b42818ea6436565c76783` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.138661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc0b5ec7f2b42818ea6436565c76783)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.141187 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.143405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/groups/bcc0b5ec7f2b42818ea6436565c76783 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:11.148959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:11.236700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 227/451] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:25:11.242150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:11.257073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95301ab5-4832-4605-a5e9-ddcab29a7da1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.270435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:11.275747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:11.295468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.296981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.297372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.297664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.298564 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] RBAC: Adding query filter params (group.id=bcc0b5ec7f2b42818ea6436565c76783, role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=b4631908c3a341389e87c50c9ca6a7c4) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:11.298880 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.300375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.306632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 225/452] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:11 2022] GET /identity/v3/role_assignments?group.id=bcc0b5ec7f2b42818ea6436565c76783&scope.project.id=b4631908c3a341389e87c50c9ca6a7c4&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:11.355898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411f511-4d11-499e-a14b-beec6a7cd85f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.395611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.396506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.396793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.397070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] PATH_INFO: `/v3/projects/5c8bf372028942759324fafea37a5438` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.398325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] RBAC: Authorizing `identity:get_project(project_id=5c8bf372028942759324fafea37a5438)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.399817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.401618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 228/453] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:25:11 2022] GET /identity/v3/projects/5c8bf372028942759324fafea37a5438 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:11.407509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:11.528368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f06f3-fb6a-4cc6-87b3-bcc727a660fe None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.565695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.566578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.566874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.567159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] PATH_INFO: `/v3/groups/7517cec076ce4cd8b1ab5b03b306144a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.568381 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] RBAC: Authorizing `identity:get_group(group_id=7517cec076ce4cd8b1ab5b03b306144a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.569891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b025314-0e48-4498-9370-b94ed9af3014 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.571549 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu May 19 20:25:11 2022] GET /identity/v3/groups/7517cec076ce4cd8b1ab5b03b306144a => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:11.578137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:11.682902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:11.689600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ba2b-9430-4196-b7fc-00966299bbdd None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:11.727108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:11.727765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:11.728049 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:11.728444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:11.729321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] RBAC: Adding query filter params (group.id=7517cec076ce4cd8b1ab5b03b306144a, role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=5c8bf372028942759324fafea37a5438) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:11.729631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:11.731072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14141eb3-1d46-44ee-8013-9cc5920ceb0a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:11.737371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 229/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu May 19 20:25:11 2022] GET /identity/v3/role_assignments?group.id=7517cec076ce4cd8b1ab5b03b306144a&scope.project.id=5c8bf372028942759324fafea37a5438&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:14.943510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73590d-8d77-4972-a9e4-01c30067ae17 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:14.944071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73590d-8d77-4972-a9e4-01c30067ae17 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:14.944071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73590d-8d77-4972-a9e4-01c30067ae17 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:14.945900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 227/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:14.957412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e08-0cec-4272-a244-4440f0beaa9a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:14.957412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e08-0cec-4272-a244-4440f0beaa9a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:14.957412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f0e08-0cec-4272-a244-4440f0beaa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:14.993666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-904f0e08-0cec-4272-a244-4440f0beaa9a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:14.995252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-904f0e08-0cec-4272-a244-4440f0beaa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:15.159128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 230/457] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:15.181581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef318fbd-ea53-429d-833c-ea6dca4aa04f None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.182262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef318fbd-ea53-429d-833c-ea6dca4aa04f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.182769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef318fbd-ea53-429d-833c-ea6dca4aa04f None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.184476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 228/458] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:15.201654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee538-5a3d-4187-8af8-b30d29967deb None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.202337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef318fbd-ea53-429d-833c-ea6dca4aa04f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.202724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee538-5a3d-4187-8af8-b30d29967deb None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.202866 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ee538-5a3d-4187-8af8-b30d29967deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.231833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-285ee538-5a3d-4187-8af8-b30d29967deb None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:15.233131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-285ee538-5a3d-4187-8af8-b30d29967deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:15.308852 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:15.376302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 231/459] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:15.377918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d6d9-8e95-41f5-84ab-ca8e00e8cf33 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.378398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d6d9-8e95-41f5-84ab-ca8e00e8cf33 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.378829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d6d9-8e95-41f5-84ab-ca8e00e8cf33 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.384302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 232/460] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:15.385657 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.386119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.386560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.403165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef318fbd-ea53-429d-833c-ea6dca4aa04f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:15.418206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:15.419823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:15.444205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:15.448597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:15.449813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.450261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.450687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.464990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:15.467118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:15.487899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:15.489178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:15.490329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:15.490329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:15.490329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.490329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:15.497175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 229/461] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:15.498625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.511410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 233/462] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:15.512485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.616678 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359af8-5a68-43b5-9a2c-5d386cb5ec2c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:15.618777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:15.627956 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5ad7e-8282-4467-8f5a-76cddb4364e7 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:15.655971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:15.657294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.657851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.658291 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.668758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:15.669362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.669652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.669932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.670855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:15.671349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:15.672969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:15.673359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:15.674828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:15.687342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 234/463] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:15.688574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:15.699548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:15.700785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 230/464] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:15.702814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.708925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:15.810212 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ddffe-5bba-443c-8fc1-7fb6ffd6b0c4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:15.823821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ca8e3-b483-4316-936f-7fcd7ca6b2c1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:15.848437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:15.848650 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.848875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.849263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.861099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:15.862553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:15.875551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:15.876241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.876568 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.876854 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.877582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:15.879157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ba222-7376-4299-8d24-2025cb777534 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:15.881845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:15.882817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:15.883806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:15.883806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:15.883806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:15.883806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:15.887926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 235/465] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:15.888881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.902967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 231/466] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:15 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:15.906420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be327c-e24a-43d4-b5e1-c7495d0e3a0a None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:15.906716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be327c-e24a-43d4-b5e1-c7495d0e3a0a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:15.907001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be327c-e24a-43d4-b5e1-c7495d0e3a0a None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:15.908420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 232/467] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:15.909185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be327c-e24a-43d4-b5e1-c7495d0e3a0a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:15.933338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:15.983943 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:16.046631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869690-0181-4429-a716-9345f8769df4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.078710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be327c-e24a-43d4-b5e1-c7495d0e3a0a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.094165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.095340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.095895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.096390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.097376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:16.097925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.099793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dec79-7214-4d31-af7b-cc3490020ba8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.114885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 236/468] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:16.116928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.117539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.118248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.119335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.120083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.120481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.120838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.121652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:16.121987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.123378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.142187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:16.143397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:16.145505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 233/469] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:16.146155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.239077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 237/470] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:16.240381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.246004 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:16.303854 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c86411-6f17-43a6-9720-18f31ce872a6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.322172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:16.345644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.346819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.347037 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.347455 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.348287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.349902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.374821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 234/471] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:16.375897 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.380981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87978ee-73d7-49df-bde1-4e36dc662d9e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.420549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.421676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.422183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.422711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.423710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.425573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.452919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 238/472] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:16.544177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1431e-6e5d-42ce-855f-aa13eb4fd63e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.578206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:16.589824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.590666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.590959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.591244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.614800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.616379 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-07b7d4df-49b0-4aa4-9d2d-6f116bf418c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:16.637047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:16.638100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:16.639571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:16.639571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:16.639571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:16.639571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:16.639571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 235/473] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:16.646341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.711276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c119b61-625d-42c8-9d77-3395ca7ea1d1 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.711580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c119b61-625d-42c8-9d77-3395ca7ea1d1 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.712014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c119b61-625d-42c8-9d77-3395ca7ea1d1 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.713760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 236/474] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:16.720231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.720611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.720965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.750686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:16.752777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:16.782683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601189b4-6881-4a71-a145-15971f30a16c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.793122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:16.836324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:16.836992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:16.837308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:16.837590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:16.838389 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:16.838705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:16.840095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:16.865081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 237/475] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:16.872604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 239/476] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:16.874740 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.895857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:16.988318 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e9b23-cd5f-4d4d-9ab9-cb493945b0b4 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:16.999260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:17.023225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cc8be-9dbb-4630-a87f-5b98f8c4ccfe None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:17.033785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:17.034831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:17.035632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:17.036436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:17.037400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:17.039781 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:17.068060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 238/477] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/users => generated 249 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:17.090061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:17.090734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:17.091032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:17.091313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:17.092104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:17.094589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:17.106783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.sql.core [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:25:17.106783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:25:17.106783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [parameters: {'id': '68e9372622d14225b5421fa5d708cb67', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:25:17.106783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application conn.query(q) May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result.read() May 19 20:25:17.128649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:25:17.129892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application next(self.gen) May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application yield resource May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application next(self.gen) May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise self.value May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application session.commit() May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:25:17.131019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.session.flush() May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application compat.raise_( May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:25:17.132008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:25:17.134473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application conn.query(q) May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result.read() May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:25:17.140733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [parameters: {'id': '68e9372622d14225b5421fa5d708cb67', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:17.142902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:25:17.145164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:17.147371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 240/478] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:25:16 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:17.147517 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:17.324865 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a324639-29d4-4467-a11f-3a364bb35505 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:17.365054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:17.365792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:17.366142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:17.366425 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:17.378179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-c4c6bd11-91d2-4579-a00c-2ad318831489 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:17.387114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:17.388217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:17.389324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 239/479] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:17.394299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:17.440291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:17.529272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a449539-2784-4eb3-b6e9-8e63acb2688f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:17.571044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:17.572044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:17.572543 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:17.573034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:17.574002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:17.574495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:17.576074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6597eb-8735-4c4f-89fa-18e5dc0dd24b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:17.586622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 241/480] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:18.588987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047f13-5e98-4048-9a94-ca0c756965b8 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:18.589558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047f13-5e98-4048-9a94-ca0c756965b8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:18.589558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047f13-5e98-4048-9a94-ca0c756965b8 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:18.591418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 240/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:18.597712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:18.598176 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:18.598481 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:18.620908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:18.622111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:18.727219 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 242/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:18.765401 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047f13-5e98-4048-9a94-ca0c756965b8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:18.807470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:18.965860 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047f13-5e98-4048-9a94-ca0c756965b8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:19.021983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:19.023091 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.023580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.024033 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.046261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:19.054960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:19.056415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:19.057668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:19.060307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 241/483] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:19.066294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:19.112854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:19.224495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484fd85-1013-48a2-92b7-ede75c8e8d6d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:19.286443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:19.287437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.287892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.288334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.289306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:19.289786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:19.291432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a957c2-8753-4bd5-a62f-ba4fb1e9b320 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:19.304657 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 243/484] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:19.316361 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:19.452772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:19.477946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15992403-dacc-43b9-b67f-ad1407487a40 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:19.485891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499a54-fda0-4408-a384-a082cd1ed2a4 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.486551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499a54-fda0-4408-a384-a082cd1ed2a4 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.487062 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499a54-fda0-4408-a384-a082cd1ed2a4 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.488731 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 244/485] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:19.502330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.502675 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.502919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.525612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:19.526427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.526716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.527003 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.540727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:19.542232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:19.576697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:19.578008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:19.590619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:19.592030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:19.593652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:19.593652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:19.593652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:19.593652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:19.593652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 242/486] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:19.599083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:19.721779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 245/487] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:19.755669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa7348-ae04-4f8a-a333-98372e265e2f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:19.759118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:19.798228 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:19.819299 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:19.819988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:19.820285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:19.820598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:19.821826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:19.822149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:19.823715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:19.851463 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:19.853853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 243/488] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:19.869216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:19.976160 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a82b5-93cb-4a65-a1f4-a89111a77bfe None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.014801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1ffa-3274-45e0-9604-a386ab5c80b8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.016177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.017057 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.017351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.017636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.030993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-f2fe4c54-884c-456b-9588-7707b0c34dbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:20.048721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:20.049849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:20.051305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 246/489] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:20.052106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcb59f-109b-4782-87d6-ffcfe296cf37 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.052106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcb59f-109b-4782-87d6-ffcfe296cf37 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.052536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcb59f-109b-4782-87d6-ffcfe296cf37 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.056434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 247/490] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:20.059688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcb59f-109b-4782-87d6-ffcfe296cf37 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.062263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.063083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.063368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.063649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.072443 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.074145 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-36973104-a477-4a79-a5a1-a8431b90f010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:20.092267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:20.093762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:20.094856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 244/491] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:20.095590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.095590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.096356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.115311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:20.116585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:20.159601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.180837 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.185900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcb59f-109b-4782-87d6-ffcfe296cf37 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.208768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 245/492] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:20.210033 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.223856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.224603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.224899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.225186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.225961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:20.226339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.228002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.242698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 248/493] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:20.251289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.353887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ebdce-75f0-4c74-9da6-cc833ca6f187 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.367181 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1f5c-40a0-4538-b57a-fd7e407a011a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.394193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.395055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.395552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.395878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.396914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:20.397337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.399074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428030c-088d-4212-869d-083f3a5d0699 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.407050 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.408002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.408364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.408743 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.417943 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.425303 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.426898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.443801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 246/494] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:20.445887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842f2b-5bbd-4de5-8f65-16c5b006605e None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.446368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842f2b-5bbd-4de5-8f65-16c5b006605e None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.446812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47842f2b-5bbd-4de5-8f65-16c5b006605e None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:20.446948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:20.448059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:20.449328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:20.449328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:20.449328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:20.449328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:20.455113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 249/495] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:20.455113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.455302 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 247/496] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:20.456237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842f2b-5bbd-4de5-8f65-16c5b006605e None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.516526 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.580523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550140ee-6dfc-47ed-a9a4-6c78ac656784 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.588244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47842f2b-5bbd-4de5-8f65-16c5b006605e None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.618917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.619770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.620065 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.620349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.631992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:20.635087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.635799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.636113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.636394 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:20.640039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:20.641167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 250/497] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:20.645355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.698237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.722321 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.723877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3596-02fe-47ff-bb12-880297c5d4f7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.753728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5625-7877-4abc-ad3a-70c21ea0cbec None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:20.771698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.795245 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:20.797003 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.797300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.797577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.798316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:20.798636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:20.800136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94876e3-cf41-42f6-96d1-477abc4e6b7f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:20.813565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 251/498] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:20.815158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.815455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.815739 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.833699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:20.834916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:20.841872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 248/499] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:20.844520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feb330-cd22-4183-bd98-57251f346250 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:20.844820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feb330-cd22-4183-bd98-57251f346250 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:20.845109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90feb330-cd22-4183-bd98-57251f346250 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:20.846615 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 249/500] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:20.848820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feb330-cd22-4183-bd98-57251f346250 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.913364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 252/501] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:20.914233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:20.967710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:20.990426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90feb330-cd22-4183-bd98-57251f346250 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.025925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.026510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.026796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.027159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.027770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313a949-5a83-4eba-805d-ccc6bdf1ee75 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.027963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:21.028285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.029822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.039125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 250/502] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:21.039981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.049418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.064839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.065644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.065928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.066206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.073465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.074909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-207016ca-521b-408c-b7ba-0f63a17a5962 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:21.092040 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:21.093822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 253/503] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:21.094903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.095942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.095942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.115258 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:21.116632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:21.162205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffa2ae-95e3-45b6-8dd6-15b36c2f98d6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.188340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 254/504] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:21.189897 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.202522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.203477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.203772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.204054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.218143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.219641 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.228827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:21.237628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:21.238886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 251/505] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:21.239937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.303965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.310515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2f7a0-51df-4ea2-a79a-8d06b33ddd8e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.346683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.347507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.347794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.348079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.348512 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd91e3-1162-401c-a88e-afd59f188ed3 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.360309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:21.368615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:21.370011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 255/506] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:21.370946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.387373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.388178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.388496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.388782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.403931 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.405543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:21.423667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:21.424929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:21.426044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:21.426044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:21.426044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.426044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:21.426044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 252/507] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:21.426360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.469167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.480512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd9a2f-886a-4db0-99f0-30b10eefa657 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.517616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.518323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.518591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.518877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.519729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:21.520056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.521555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.535946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79adc1-4c10-4d53-a4a1-b06108fbcac7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.547378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.561017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 256/508] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:21.562356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.576264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.577150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.577428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.577707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.588810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.590290 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:21.614075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:21.615248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 253/509] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:21.616721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.669072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925b9b5-4228-4bca-b390-a6c17d77d456 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.704976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.705661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.705962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.706245 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.706998 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:21.707318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.708818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.721855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 257/510] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:21.722826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.727947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32691c-65e1-4376-ac4b-88df5cd0b737 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.735694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.764308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.765067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.765414 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.765702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.766480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:21.766796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.768221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.777039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 254/511] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:21.777850 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.805295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.838317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a20b9-ffbf-40fd-9565-bc3c64c05a47 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.875391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.876440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.876440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.876668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.877392 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:21.877712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.879113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:21.888294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6169-241b-48cc-8cdc-3991b78851c5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:21.891762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 258/512] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:21.892627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:21.926544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:21.927272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:21.927563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:21.927844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:21.962741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:21.999260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:21.999564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d884ad7-8f46-42ee-895b-f0c390348a3e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.000920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.027436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.044392 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.045093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.045906 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.046183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.047074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:22.047388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.048812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.060193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 259/513] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:22.061649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.110324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 255/514] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:22.111181 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.172378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe894d46-2194-40b2-9146-f980a1421f27 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.200369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.209520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.210334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.210628 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.210916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.218406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.219866 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.224916 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25daa9-4f91-444a-849a-673e979e1909 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:22.236293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:22.237539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:22.238561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 260/515] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:22.239157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.255698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.267016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.267824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.268113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.268420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.288166 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.289725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:22.309297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:22.310405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 256/516] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:22.312157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.352680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f404c2e-bc77-4b5a-94cb-4392db584b0e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.397325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.398136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.398425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.398705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.406814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.408289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:22.424269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:22.425549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:22.426595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:22.427194 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca1aafe-278f-4e6c-a5c9-ee9d059c5a28 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.427282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 261/517] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:22.427282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.471891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.472607 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.472903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.473188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.473905 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.475369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.480907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.499369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 257/518] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:22 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:22.500312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.549635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.551228 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1dac7-1df3-4219-96dc-252f4649d78b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.608640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.609495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.609847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.610132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.621542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.623058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.641705 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888e3a2-2c64-4117-8e2a-522ea42012e0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:22.643272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:22.644289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:22.645365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:22.645365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:22.645365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:22.645365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:22.645365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 262/519] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:22.645736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.689331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.689919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.690201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.690483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.691331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:22.691664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.693246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.713435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 258/520] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:22.714303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.777946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.781917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cea2a1-0540-4044-a439-4a9d19fd9a77 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.824719 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.825399 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.825696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.825976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.826755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:22.827071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.827717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:22.828488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.837270 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce120d8-e8d9-4a77-af07-45c822bea96b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:22.844561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 263/521] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:22.844561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.896457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:22.897162 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:22.897451 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:22.897739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:22.898600 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:22.898913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:22.900596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:22.923692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 259/522] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:22.924561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:22.975535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e4a4f-4f82-4883-9416-ac5749823a13 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.011646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.013288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.013587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.013868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.015451 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:23.015766 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.017490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.039600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 264/523] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:23.039600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.051503 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b319a-8e34-4994-b702-2680fb7078b8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.067288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.090977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.091640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.091925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.100306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.100306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:23.100306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.100306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.106692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 260/524] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:22 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:23.106692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.126171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.180797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6c935-5cf6-480c-a6da-42e2715d22e6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.223288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.224505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1bf46-3dd2-4d40-bb1a-a55430c7444a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.224937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.225437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.225997 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.234883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.236869 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:23.254413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:23.255536 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:23.256674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:23.259566 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 265/525] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:23.261182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.263617 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.264556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.264849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.265195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.346444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.347956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.365218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.371431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.394403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663ca8f-0a52-41e0-8392-061c696c27f0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.438581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.439809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.440194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.440606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.448455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.449894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:23.465684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:23.466920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:23.468016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:23.477071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 266/527] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:23.477071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.477451 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 261/527] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:23.478650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.587540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b1a35-7fe8-4785-a412-1ee9766b8825 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.591011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ff3ac-d567-4cb0-8bf9-ed159bb23a22 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.604176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.625793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.626620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.627007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.627501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.628523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:23.629006 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.630596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.630990 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.631909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.632385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.632887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.634013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:23.634594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.636273 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.648514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 262/528] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:23.648514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.652828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.668317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 267/529] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:23.669412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.760817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659de396-7abe-47e5-8679-b843e3fc8b7b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.788980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd6ced-c116-4cd9-a41a-e8997e7eb6b8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:23.799984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.801193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.801652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.802106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.815058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.816920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.828501 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:23.829274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:23.829632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:23.829986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:23.830892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=57a2d995f2534ff8a37af279bf6fec2c) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:23.831276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:23.833019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:23.838164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:23.839528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:23.840861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:23.840861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:23.840861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:23.840861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:23.847400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 268/531] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/role_assignments?user.id=57a2d995f2534ff8a37af279bf6fec2c&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:23.847400 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.848166 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 263/531] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:23.849364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:23.883651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.943494 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:23.990153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88c38a-6a91-4565-b407-8a8ce6e80267 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.035459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca07966-f12c-4bc3-9f75-49ebfe2ec763 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.042056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.042815 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.043175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.043530 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.120971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.121664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.122009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.122300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.123054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:24.123374 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.133808 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.145199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.147398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.159752 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 264/532] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:24.165577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:24.246917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:24.328118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:24.334380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 269/533] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:24.341105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:24.366479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcba053-2e12-4b56-8fcb-bf5bcbff3853 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.444203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.445057 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.445347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.445626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.456205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.457727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:24.474123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:24.475174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:24.476204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 265/534] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:24.482148 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:24.511844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f013e-6e3a-4eeb-8828-8652721a1ffa None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.556303 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.557196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.557683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.557973 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.577200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.578965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:24.601840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:24.604060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:24.606281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:24.606281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:24.606281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:24.606281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:24.606281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 270/535] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:24.613572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:24.649124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:24.748429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820309a-c196-4ca8-b6cc-39fe80ef137f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.791836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.792579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.792877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.793172 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.794015 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:24.794336 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.796713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.816132 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 266/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:24.821992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:24.844873 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:24.850538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc8efd-483b-48c3-bcba-bfc6e0e67b34 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:24.895689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:24.896756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:24.897944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:24.898576 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:24.900723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:24.901945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:24.903620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:24.929105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 271/537] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:24.937806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.026493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:25.056341 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96b36-8c9f-4b9b-b9eb-9d2195c7f563 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.057721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c63ee-59b2-4ac3-b90a-8473e4aecf06 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.100834 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.101564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.101847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.102122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.110291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:25.111722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:25.112876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.114200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.115061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.115630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.116800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=8b77128461364caebdcab9b93b2aaa5d) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:25.117353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:25.119972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcb245-b96d-4467-81a7-8778cd62d999 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:25.127616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 267/538] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/role_assignments?user.id=8b77128461364caebdcab9b93b2aaa5d&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-662943a9-75ba-417c-b75c-49dbbf6650a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:25.128705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:25.129691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:25.131113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 272/539] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:25.132731 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9d19-9010-4283-8787-c1096b3146b6 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.133259 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9d19-9010-4283-8787-c1096b3146b6 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.133758 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b9d19-9010-4283-8787-c1096b3146b6 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.136772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 268/540] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:25.145189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.145189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.145189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.145590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b9d19-9010-4283-8787-c1096b3146b6 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.174828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:25.176369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:25.324438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:25.355875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 273/541] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:25.386857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.397243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b9d19-9010-4283-8787-c1096b3146b6 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.459533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.461561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.461856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.462203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.463031 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:25.463351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:25.464946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:25.494912 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 269/542] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:25.501060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.503743 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:25.554958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45089b44-c6a5-4c73-9ca5-2339d23433a2 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.618462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.619668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.620172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.620694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.636599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:25.657437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:25.658552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:25.659582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:25.669345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 274/543] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:25.675863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.692905 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854927eb-af2f-41c2-ba4f-54521187f3e2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.731799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:25.737417 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.738103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.738395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.738680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.739541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=fac286d2b5514467be990cbea641cb5b) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:25.739861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:25.741364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:25.760816 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 270/544] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:25 2022] GET /identity/v3/role_assignments?user.id=fac286d2b5514467be990cbea641cb5b&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:25.833515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27a25-54cf-4900-b4ee-769df0efcdfb None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:25.864832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:25.879146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:25.879857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.880155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.880463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.881265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:25.881586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:25.883046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb4183-fc53-469c-8fdb-a24f088112cf None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:25.893993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 275/545] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:25.901628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:25.935664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a4c2-a722-4a01-bda9-8d4ae59eef31 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.936061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a4c2-a722-4a01-bda9-8d4ae59eef31 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.936313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a4c2-a722-4a01-bda9-8d4ae59eef31 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.937837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 276/546] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:25.944007 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:25.944230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:25.944530 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:25.997573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:25.999405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:26.045473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df156cb2-4f6f-47da-97b0-66808b7c086d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.083055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.083945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.084236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.084596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.097359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.098910 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:26.117956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:26.119029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:26.120028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:26.120028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:26.120028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:26.120028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:26.120028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 271/547] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:26.126864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.164018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 277/548] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:26.212378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.216075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:26.317950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfede702-892b-44bd-a56f-bfd4bbf2589e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.345840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2bf4d7-38a1-4dd8-af83-59aa7cdb5c2a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.367753 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:26.388278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.389752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.390711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.391923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.405879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:26.414419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:26.415718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:26.416911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:26.419337 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 278/549] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:26.425802 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.433140 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.433843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.434141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.434427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.435178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:26.435495 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.437537 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:26.471256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 272/550] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:26.481422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.548584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cffe2-7b1d-427e-9837-f2399731df50 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.586646 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.587630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.588087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.588583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.589488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:26.589964 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.591516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:26.602730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 279/551] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:26.608772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.684903 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:26.688946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bd330-a59e-4278-b35a-0c8d72bc4537 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.705267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:26.767397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.768219 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.768540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.768824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.777788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.779578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:26.797017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:26.798118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:26.799176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 273/552] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:26.804420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.841601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e5492-7fb7-4cab-bbac-2d6f3dbd5ad1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.915091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.916492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.916936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.917490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.930919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.932476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:26.951145 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811a66a-ad2a-4142-bc35-699169536e9f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-ade6e2c6-7ac7-40c1-aba7-5eee9d6151dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:26.952109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:26.953187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:26.954186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:26.954186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:26.954186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:26.954186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:26.954186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 280/553] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:26.960431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26706f-73d6-43ce-b883-6ce4438237dc None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.960431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26706f-73d6-43ce-b883-6ce4438237dc None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.960431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26706f-73d6-43ce-b883-6ce4438237dc None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.968672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 281/554] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:26.969841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26706f-73d6-43ce-b883-6ce4438237dc None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:26.994256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:26.994940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:26.995228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:26.995512 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:26.996355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:26.996695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:26.998149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb4ce-2376-48f4-99d0-7c2c804d9cc4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.017286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 274/555] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:27.018676 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.018963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.019249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.041580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:27.042783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:27.062586 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.098257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.112182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26706f-73d6-43ce-b883-6ce4438237dc None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.124470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 275/556] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:27.124470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.187342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.188502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.189458 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.189921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.191555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:27.192056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.194622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.211791 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 282/557] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:27.213013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.253341 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576a04f-7007-4ca4-a61c-596a8a985208 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.306128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.306128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.306128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.306128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.343734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.357231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9081c-81f6-431a-91fe-d5a581d1d841 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.397966 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.399220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.399685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.400135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.403214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.405202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.413240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:27.423201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:27.424547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:27.425544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:27.433193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 283/558] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:27.434611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.450305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.532643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 276/559] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:27.534017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.555314 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e69ba-9d7c-4524-a16d-b06aad662480 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.601420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.602591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.603111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.603584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.604907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.616912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.618805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:27.642796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:27.644022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:27.650358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:27.653572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 284/560] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:27.654828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.662810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17ecce1-75c6-4965-9620-b6c9d231644a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.714898 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.715870 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.716160 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.716463 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.717773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:27.718088 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.720300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.732113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 277/561] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:27.733609 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.748565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.793806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1fcf6-9021-47c4-b246-d3744cc67180 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.842817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.843754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.844044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.844320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.865140 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622529e-1619-42f5-ac6b-96da990ea09e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:27.869468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.871034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.885050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:27.891873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:27.893017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 285/562] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:27.894017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:27.914050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:27.914733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.915133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.915419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.916422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:27.916756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:27.918335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b75ab1-bd46-4efb-a733-1c2517eee947 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:27.944314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 278/563] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:27.944314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a1dd-097c-44a3-a079-ea7905e0e4e8 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:27.944314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a1dd-097c-44a3-a079-ea7905e0e4e8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:27.944314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21a1dd-097c-44a3-a079-ea7905e0e4e8 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:27.944314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 279/564] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:27.944848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21a1dd-097c-44a3-a079-ea7905e0e4e8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.038024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafc434-75b1-4312-be57-e96121e12edd None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.067541 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.067933 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21a1dd-097c-44a3-a079-ea7905e0e4e8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.081917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.082733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.083021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.083300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.095268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.096803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.112526 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.113272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.113567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.113834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.114589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:28.114931 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:28.115097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:28.116056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:28.117776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:28.117776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:28.117776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:28.117776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:28.118029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d00f613-63aa-454e-8b85-8d54bf1d6c7c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.118123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 286/565] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:28.118123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.129557 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 280/566] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:28.130670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.130962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.131243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.152284 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:28.154104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:28.189519 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.239784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fe5b2-588c-405b-b3a6-1234479dbfbb None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.241044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 281/567] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:28.242044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.276353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.277189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.277477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.277759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.289407 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.290875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:28.308350 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:28.309648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 287/568] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:28.310899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.341451 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.361355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439fbd-ecbb-420e-9e2c-055ea885065f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.398278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.398911 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.399247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.399535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.400279 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:28.400620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.402016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f47f0-b0b3-4479-ae65-b03b2f88d5fb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.414028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 282/569] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:28.415328 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145aa9-a227-4b2b-abe2-860cca027385 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.415630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145aa9-a227-4b2b-abe2-860cca027385 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.415919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31145aa9-a227-4b2b-abe2-860cca027385 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.417197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 283/570] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:28.417913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31145aa9-a227-4b2b-abe2-860cca027385 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.423069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5704d5-88cb-4185-b0e8-d2e8e2a20604 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.449861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.461429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.462239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.462534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.462819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.469926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.471363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:28.488036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:28.489352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 288/571] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:28.490375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.538900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31145aa9-a227-4b2b-abe2-860cca027385 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.573578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.574467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.574762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.575051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.584517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.588070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:28.596198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:28.597569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 284/572] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:28.598721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.605123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece345f0-fa4e-4966-9207-cd9fe99be725 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.643839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.644556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.644847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.645728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.645851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:28.646168 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.647560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0277558-517e-4855-85ff-8dc33f111a06 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.659979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 289/573] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:28.661308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.661609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.661893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.678911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:28.680274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:28.711758 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.729209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a28eb-3171-4b29-b5d0-2f0eec9c7ac5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.754629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 290/574] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:28.755676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.776438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.777160 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.777410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.777692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] PATH_INFO: `/v3/users/glance` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.788364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.791013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:28.807700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:28.809053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 285/575] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:28.810208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.851296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:28.876784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8eb9f-f919-4af6-99c3-26574b2d943a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.914097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.914777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.915068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.915351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.916187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:28.916547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.917990 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.921370 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580267d-8abc-4278-8398-e0ae433b429c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:28.935883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 291/576] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:28.936881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.959153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:28.959831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:28.960124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:28.960431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:28.961212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:28.961536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:28.962930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:28.971914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 286/577] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:28.972752 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:28.981777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.050669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea37586-035a-4970-8a74-cffc17489e3f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.080719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482e54-37fd-4619-a0f5-a2ff2d3e16f2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.084264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.085175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.085466 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.085746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.088560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.094123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.095626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:29.110898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:29.112091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 292/578] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:29.113363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.131289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.132111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.132419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.132825 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.143907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:29.152169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:29.153650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 287/579] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:29.154573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.226189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.230578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00e588-c6ad-4ab2-a864-f30ff7e6bb25 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.265281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762d838-4454-4058-ac41-294bf13ee4e4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.270409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.271088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.271375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.271654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.272542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:29.272865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.274311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.295096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 293/580] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:29.295989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.302125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.302793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.303081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.303359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.304221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=5df2284d1c2c4e3d81010000420d8646) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:29.305125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.306637 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.313947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 288/581] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/role_assignments?user.id=5df2284d1c2c4e3d81010000420d8646&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:29.315135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.379260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.434556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9912cd4-d973-43f7-866c-88dc879082e8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.438839 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafd70-e420-41be-9f0e-347096d00ad3 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.479351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.480126 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.481061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.481448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.481701 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.487443 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.488144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.488512 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.488769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.489596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:29.489914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.491459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.494737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.496240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.506452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 289/582] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:29.506708 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:29.519648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:29.521916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 294/583] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:29.523260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.634810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15af9-bbff-4389-ba81-375258148dfa None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.636069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e56-6b27-4c8e-a7cc-ceadb53d947e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.643719 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.680420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.681124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.681418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.681544 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.681776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.682278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.682469 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:29.682591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.682786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.682888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.684187 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.694852 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.695958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 290/584] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:29.696326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.701181 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:29.716475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:29.718281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 295/585] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:29.720166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.778745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:29.830349 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4f7a-eb78-4800-aa4a-2cc155f396af None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.843715 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424cbf04-ac31-4a7c-87d6-753e1ced4d77 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:29.875306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.876295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.876644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.876934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.890111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:29.890959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:29.891314 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:29.891660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:29.892518 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:29.892898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.894614 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.895404 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=d4759f19e262476a8c4c4a99b43daa13, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:29.896987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:29.913018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 296/586] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:29.913018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.924541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 291/587] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:29 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:29.924541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:29.939707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.052994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590fc45-bc02-4bb0-a31f-b4f1e88eebcb None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.091557 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1c476-3ff1-41ce-a556-a2c8820f87c5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.094213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.096962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.097737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.098134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.098523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.112582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.114384 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:30.142856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:30.143912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:30.144944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:30.144944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:30.144944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:30.144944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:30.149074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 297/588] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:30.149074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.159098 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.159795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.160139 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.160499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.161879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:30.161879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.163067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.178117 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 292/589] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:30.178935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.250584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.294330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ef980-7d10-4a98-9e24-122458635713 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.300311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926a3c1-e601-40af-8646-af0440b08872 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.343966 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.344253 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.345143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.345493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.345619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.345990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.346201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.346626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.347028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:30.347452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.349813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.356773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.358624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.366236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 293/590] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:30.367539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:30.376860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:30.378213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:30.379476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:30.386318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 298/591] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:30.387515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.401955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.512781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4196cb-6716-4c8b-8f44-939880c00364 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.517786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74869337-cf43-4a38-9a75-9f65978d9bfb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.547094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.556360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.557524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.558097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.558533 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.568907 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.570075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.570370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.571283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.571675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.572326 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.580984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.582598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:30.586132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:30.587381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 299/592] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:30.588783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:30.603069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:30.604390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:30.605573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:30.609165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 294/593] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:30.609165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.721106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.740284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af29f2-0a53-49d2-9559-1dda50b9e988 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.744769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12dd1-9717-4a46-b5f3-6f69ede061cb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.788373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.789291 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.789653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.790104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.790724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.791108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:30.791496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.791606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.791962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.792313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.793228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.793340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:30.793711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.795274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.815024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 295/594] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:30.816069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.816619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 300/595] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:30.820147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.877149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.931538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa3705-efa8-4e0c-88ce-8761d2bd7af0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.936941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec69859-87cf-46e2-be6a-dd15c3554074 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:30.970816 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.971865 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.971986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.972190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.973184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:30.973425 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.975060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:30.978332 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:30.979428 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:30.979928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:30.980445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:30.986685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:30.991641 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 296/596] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:30.993365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:30.994389 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:30.996153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:31.015634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:31.016722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:31.017798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:31.017798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:31.017798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:31.017798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:31.020181 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 301/597] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:31.021480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.117879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29b147-0d98-496b-a1e7-9f412f319afc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.153696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:31.158689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665822d9-31eb-4daf-b616-3ade7be9285e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.163308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.164449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.164963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.165458 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.178560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.180388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:31.202400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:31.203664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:31.204672 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:31.204672 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:31.204672 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:31.204672 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:31.207284 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 297/598] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:31.208596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.214110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.215111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.215485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.215847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/57a2d995f2534ff8a37af279bf6fec2c/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.236463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=57a2d995f2534ff8a37af279bf6fec2c, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.238597 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.258349 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 302/599] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:31 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/57a2d995f2534ff8a37af279bf6fec2c/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:31.259959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.276206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:31.410625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df66b4-1b13-4a25-b057-c59c8f8ffca9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.417089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b0448-37ec-481f-8643-9482472fec23 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.467770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.468563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.468960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.469248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.470055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:31.470379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.471779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.485558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 303/600] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:31.497540 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.538319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.538933 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:31.539703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.539929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.540427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.541767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:31.542300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.544890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.580385 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 298/601] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:31.587133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.661995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020e0bb-9493-4664-9efd-53a1fc21ad8a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.723319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5a6a5-dc56-4bce-a7be-95e47ed4f7b7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.739756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:31.768349 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.769685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.769915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:31.770436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.770785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.771077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.771357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.771566 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.784266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.787984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:31.789679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.791328 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a9e8bf66-ad38-4edc-b986-1c5e0573a885 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:31.818251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:31.819530 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:31.820790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-6d5cb11f-41d5-401a-8984-88f39fbaed9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:31.821529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:31.822647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:31.823884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:31.825275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 299/602] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:31.832532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52784-4fce-4765-9cb8-a3146657ef3e None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.832532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52784-4fce-4765-9cb8-a3146657ef3e None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.832532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b52784-4fce-4765-9cb8-a3146657ef3e None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.833276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 304/604] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:31.833403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 300/604] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:31.834133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b52784-4fce-4765-9cb8-a3146657ef3e None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:31.836841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:31.837426 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:31.837927 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:31.868704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:31.870963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:31.960256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:31.963405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b52784-4fce-4765-9cb8-a3146657ef3e None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:31.968978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 305/605] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:31.970018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.019674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.020652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.021130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.021582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.022555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:32.023030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.024698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.039079 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 301/606] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:32.044742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.083022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:32.107083 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382e4fb-42b2-421d-8ba2-49000630af5b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.157186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.158098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.158463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.158897 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.159787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:32.160177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.161746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.161960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71894a2d-355d-407a-90b4-4c3323de470e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.177382 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 306/607] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:32.178308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.205932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.206836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.207199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.207543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.220185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:32.228812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:32.230234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:32.231815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:32.237446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 302/608] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:32.238446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.260705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:32.318997 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2602635-8a19-49a6-97bb-a5bdb8c584bc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.350297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:32.360807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af2cd67-5b2c-4b45-a0e6-831419428a7d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.369302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.370558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.371006 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.371441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/8b77128461364caebdcab9b93b2aaa5d/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.396990 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=8b77128461364caebdcab9b93b2aaa5d, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.398705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.400035 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.401018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.401359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.401796 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/fac286d2b5514467be990cbea641cb5b/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.417520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 307/609] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:32 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/8b77128461364caebdcab9b93b2aaa5d/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:32.418778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.420264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=fac286d2b5514467be990cbea641cb5b, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.422329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c826a-0d1b-4c9d-ba11-1e57a6b2a770 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.449351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 303/610] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:32 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/fac286d2b5514467be990cbea641cb5b/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:32.457476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e0163-3f83-4172-afad-865afa918be7 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.457856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e0163-3f83-4172-afad-865afa918be7 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.458052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e0163-3f83-4172-afad-865afa918be7 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.459806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 304/611] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:32.465606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.465927 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.466165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.485857 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:32.487246 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:32.575889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3d6a9-8b3c-4660-8182-19a7a7475a73 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.621776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.622911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.623566 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.624092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.625308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:32.625865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.627500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.641949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 308/612] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:32.642339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:32.648996 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.674114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 305/613] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:32.709679 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.818405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:32.842974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e5cc-b69c-4eea-9fa6-c65872c580cc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.863779 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66746048-b7c0-4a29-bfd4-ca0725b0cd61 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:32.891254 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.892231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.892648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.893070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.912807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:32.914016 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:32.914480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:32.914912 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:32.921888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:32.923510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:32.927720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:32.937633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:32.938775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:32.939973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 306/614] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:32.944561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:32.958209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:32.959276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:32.960306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:32.960306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:32.960306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:32.960306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:32.962723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 309/615] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:32.968468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.003771 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:33.088254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778005a-b86c-4f5d-8e10-b1a0b34d4925 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.096194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c6b3-edbd-4d91-b2bf-9207a6917512 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.126405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:33.133398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.134084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.134377 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.134660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.135427 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:33.135742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.137182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.138296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.139495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.140137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.140653 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.141780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:33.142300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.143905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.146778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 307/616] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:33.153674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.171929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 310/617] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:33.178132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.291871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50941d-d972-4da8-ae83-352dbfddf584 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.340927 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.342819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.343257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.343774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.357706 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.359390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:33.386189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:33.387249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:33.388238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:33.388238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:33.388238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:33.388238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:33.388238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 308/618] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:33.395596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.416830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:33.438328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7417e7d-4096-4abe-9684-f40ae28dd9d6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.484811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.484811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.484811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.484811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.490479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:33.493721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.495379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:33.516375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:33.517512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:33.518590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:33.527415 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 311/619] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:33.533221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.594541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091ba53-5b8e-4766-8cba-fa31abd5af06 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.639380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.640161 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.640480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.640770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.641528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:33.641845 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.643259 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.672803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 309/620] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:33.679190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.742517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81287f5-3df2-4828-8267-257fde046ca0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.816172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:33.816914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:33.819963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:33.820275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:33.821202 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:33.821527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:33.823002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:33.848622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 312/621] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:33.861604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:33.867203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:33.907329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c5e84-3c90-41e9-b625-738bb970528b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:33.919388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.005672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af71d4-1ba4-441d-aa44-e27490f86f4d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.009787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.010631 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.010925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.011211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.026983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.028536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a15f5c18-8975-418b-ab17-87439699a5e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:34.044505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:34.046368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:34.047450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:34.051159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 310/622] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:34.052698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb0a4-c24d-420d-bf35-9f9c2071c910 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.053001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb0a4-c24d-420d-bf35-9f9c2071c910 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.053292 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb0a4-c24d-420d-bf35-9f9c2071c910 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.054293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.055263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 311/623] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:34.055711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.056027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bb0a4-c24d-420d-bf35-9f9c2071c910 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.056460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.056919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.070014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.071468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-095575b8-8bd1-4629-a681-299281bf7fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:34.090770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:34.092522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:34.093730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:34.093730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:34.093730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:34.093730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:34.097030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 313/624] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:34.098527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.098825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.099109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.117198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:34.118451 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:34.174696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.181896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bb0a4-c24d-420d-bf35-9f9c2071c910 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.204726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 314/625] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:34.205836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.226465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.227495 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.227982 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.228504 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.229565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:34.230045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.231742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.253550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 312/626] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:34.254889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.291316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.331593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ee74f-3ea7-42d8-8209-36c985cf8d59 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.372819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.374145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.374486 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.374764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.375641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:34.375869 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.377371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.390517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 315/627] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:34.395141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5adba-94c5-4962-8cec-58c740511f0f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.395895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.443167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.444383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.444818 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.445205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.448556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.461390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:34.472281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:34.473623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:34.474762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:34.480480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 313/628] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:34.481588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.531588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa7164-47f3-4d5b-8960-ce16b85d2a59 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.579044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.580046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.580476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.580874 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.584144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.603331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2e46b-81be-4901-a75a-00aefeca9187 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.641998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.643119 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.643546 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.643998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.651848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.653650 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.671918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:34.673556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:34.674730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:34.675970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:34.676608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.683091 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 314/629] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:34.684235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.735616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.800033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 316/630] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:34.801368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.814981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e614fd-2a4f-4a5f-be7e-5839504b6a1f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.863482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.864669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.865071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.865489 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.866783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:34.867649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.869884 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.889749 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 315/631] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:34.890960 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.901903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:34.918863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8e28-cc14-49f0-88ea-1f4f066ed842 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:34.958688 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:34.959671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:34.960109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:34.960687 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:34.961638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:34.962104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:34.963705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:34.983168 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 317/632] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:34.984523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:34.988738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.010456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2083b82-872a-42e0-bef8-facea1b34275 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.052086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.053019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.053418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.054000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.067788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.069272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:35.088351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:35.089601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:35.090810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:35.090810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:35.090810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:35.090810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:35.092639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 316/633] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:35.093611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.109947 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92189bf6-ca49-4062-a825-2e93af2980be None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.150149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.151493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.151950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.152456 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.161089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.165520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.167604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:35.188285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:35.190104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:35.191120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:35.191120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:35.191120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:35.191120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:35.194044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 318/634] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:35.195282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.220151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028c15-d242-4077-b262-dc6e6ddfcf5c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.258950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.259783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.260070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.260354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/5df2284d1c2c4e3d81010000420d8646/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.261476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.277361 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=5df2284d1c2c4e3d81010000420d8646, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.278842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.293528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 317/635] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:35 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/5df2284d1c2c4e3d81010000420d8646/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:35.294418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.337260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1077-be99-4429-abc6-0c077d245d3d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.382631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.383329 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.383651 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.383937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.384746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:35.385063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.386487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ea37c-f830-4a08-8db3-f393a379167f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.402109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 319/636] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:35.403602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3674-a67f-43e4-b599-a6d76a178c87 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.403900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3674-a67f-43e4-b599-a6d76a178c87 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.404187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb3674-a67f-43e4-b599-a6d76a178c87 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.406611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 320/637] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:35.407880 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf599e4-6506-425d-8823-8e6e33569d92 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.408175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf599e4-6506-425d-8823-8e6e33569d92 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.412444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf599e4-6506-425d-8823-8e6e33569d92 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.412444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 321/638] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:35.412444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf599e4-6506-425d-8823-8e6e33569d92 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.444038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8940255-7be9-4fa0-a8f8-d6f5ae594177 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.457118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.490266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.490958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.491252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.493413 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.493803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:35.494606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.496376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a854c6-2878-442a-ac1b-cb65b86c42b2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.511671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 318/639] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:35.512850 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.513146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.513437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.532130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:35.533434 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:35.541673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf599e4-6506-425d-8823-8e6e33569d92 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.562832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.584825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.585641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.585926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.586206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.594656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.596125 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-81fb73ce-9707-4ab9-b0ca-ca4baebffc4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:35.611953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:35.613464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 322/640] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:35.614508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.615113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.615113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.632655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 319/641] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:35.632955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:35.634145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:35.637531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.728487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 323/642] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:35.729710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.758311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.767951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825cd1d7-ec92-40b0-b54c-3ca6c4237c4c None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.807418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.808238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.808559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.808844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.818213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.819739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:35.838043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:35.839149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:25:35.840266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:35.842759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 320/643] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:35.843670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.866724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691bcf6-f7a5-454b-98ed-b0bf4af59822 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:35.881211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:35.918559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:35.919632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:35.920171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:35.921201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:35.922845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:35.923463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:35.925934 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:35.946390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 324/644] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:35.946390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:35.978923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e70e91-b558-49fc-84a4-f11bdfbcf8e9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.018394 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.019134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.019424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.019712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.033647 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:36.041644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:36.042661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:36.043589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 321/645] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:36.044346 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.067395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.069645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25931fa-ec79-4a71-85bf-82272d1e044b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.114405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.115235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.115523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.115804 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.128062 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:36.136268 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:36.137755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:36.139861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:36.142832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 325/646] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:36.143689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.169393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc59a9-a12c-413f-90e8-561438e2b87f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.172869 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.210216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.210945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.211238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.211448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.212386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:36.212630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.214252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.236925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 322/647] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:36.238189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.273406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa949b0-004f-43f8-b5b4-174dc39becce None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.319079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.319867 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.320168 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.320491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.321341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:36.321665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.323113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.340631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 326/648] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:36.340631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.346748 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.377585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027aafb9-7c4e-468e-8a23-d9b1ca2af00f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.422084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.422791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.423344 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.423626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.424916 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:36.425238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.426669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.441610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 323/649] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:36.443136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.490752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.504123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1ec27-d39a-45d9-9678-db2a78ccafce None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.550226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.550839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.551126 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.551409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.552378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:36.552830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.554961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.565485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 327/650] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:36.566375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.584649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2b9c4-ce98-4bac-a99b-3cda8c34f826 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.635202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.636230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.636824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.637335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.658263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.715353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04157bab-de9f-4c45-882b-4f583ca2b0ed None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:36.732680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.735003 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.770366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:36.771474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:36.771768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:36.772054 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:36.787423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:36.789410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:36.812645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:36.813705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:36.815070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:36.815070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:36.815070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:36.815070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:36.819264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 328/651] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:36.820252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.827496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.892344 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 324/652] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:36.893299 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:36.949703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:36.974725 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78bdc7-741f-4e80-a8fb-f4fe3d233668 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.013932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.015030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.015475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.015921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.025007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8ddab-0a5e-4d46-976d-4466b3f0cad3 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.029332 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.031031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:37.051142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:37.052455 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:37.053683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:37.053683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:37.053683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:37.053683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:37.055637 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 329/653] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:37.056805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.066536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.067294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.067691 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.068084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.069380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:37.069380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.071013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.089172 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 325/654] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:37.090304 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.136790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.181165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a600c4c-60b1-4861-9b28-dad3b4464b26 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.199561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.204337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c88b25-d25d-47ed-b7d2-bc4a7f99afb1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.222376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.223470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.223924 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.224371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.238698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.240708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.245808 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.246582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.246950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.247314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.248136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:37.248637 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.250127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:37.262127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:37.263182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:37.264796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:37.264796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:37.264796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:37.264796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:37.265060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 326/655] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:37.265146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.267266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 330/656] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:37.276468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.384306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee986fd-fb12-4599-a453-710d343ab21d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.405134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaef58-3941-463b-a1fb-377f1fb5e614 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.415338 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.425787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.427179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.427656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.428128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.440494 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.442862 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.447100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.447860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.448217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.448609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.449422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:37.449800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.452079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:37.461591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:37.462871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:25:37.464156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:37.470654 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 327/657] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:37.470654 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.476528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 331/658] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:37.477639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.492487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.605186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cad00-685f-4923-a716-0b3b8962dc10 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.620290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c701-24f8-4bef-a80e-8ddabb38d36c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.644173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.645661 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.646537 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.647049 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.657235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.659388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.664685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.665374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.665764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.666056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.666912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:37.667227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.668795 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:37.678359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:37.679445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:25:37.680523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:37.683300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 328/659] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:37.684432 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.694515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 332/660] 10.222.0.13 () {56 vars in 1137 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:37.695408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.727185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.777150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:37.804216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e41f7d-d930-49ef-8a66-84441d994b10 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.822555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c824f8c-da4f-4c19-87b6-8dd5efc3b1d6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:37.843395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.844272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.844586 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.844881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.856475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.858833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.865793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:37.866474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.866764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.867044 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.867889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:37.868206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:37.869672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a610f-13af-4ad7-a520-75693184043e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:37.873964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:37.875278 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:37.876311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:37.880045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 329/661] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:37.881144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 333/662] 10.222.0.13 () {56 vars in 1139 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5222-0889-4263-bcc4-4547c69200e8 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5222-0889-4263-bcc4-4547c69200e8 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5222-0889-4263-bcc4-4547c69200e8 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 334/663] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:37.900773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db5222-0889-4263-bcc4-4547c69200e8 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.010367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f05765-a5b0-4722-ac46-ceafeae36eb2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.011166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.033686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db5222-0889-4263-bcc4-4547c69200e8 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.048900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.049841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.050323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.050777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.070765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.073404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.074082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.074371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.074649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.075452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:38.075770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.077446 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d95501-160a-4345-a416-11c40075e3c0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.089835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 335/664] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:38.096310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.096647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.096913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.115918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:38.117179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:38.143033 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.144584 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.197287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 336/665] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:38.198464 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.268882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 330/666] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:38.270125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.291602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.316527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.330644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a1009-b0bf-4102-84b7-822ca4ecd18e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.380232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.380948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.381246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.381784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.385204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758477e-b039-4c42-a3c9-af0c5f5ffccc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.424366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.425380 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.425828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.426288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.427301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=57a2d995f2534ff8a37af279bf6fec2c) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:38.427779 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.429969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.437910 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 331/667] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/role_assignments?user.id=57a2d995f2534ff8a37af279bf6fec2c&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:38.438998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.473013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.474453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.584602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.622995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb73d45-a44e-4b2d-b6ca-ac0e19fd4523 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.622995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.634140 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 337/668] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:38.634140 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.673883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.674858 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.681093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.681702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.702765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:38.711391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:38.713497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:38.715384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:38.718219 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 332/669] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:38.719156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.793984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67597f6-b7e7-4dc7-8cf1-97f20477e7f8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.839506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.840363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.840734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.841014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.853136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.854593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.865773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4bf25-4e03-4f32-b03d-b5a297500023 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:38.872931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:38.874826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 338/670] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:38.875822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.908318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:38.910827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:38.911634 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.911920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.912203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.924464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:38.925978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-be1e1ed5-77f1-4042-8f35-127b74799840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:38.945235 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:38.946545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:38.947578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:38.947578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:38.947578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:38.947578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:38.947578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 333/671] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:38.948448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52c4d-0143-4bb9-a150-e5aa54a7b1bc None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:38.948448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52c4d-0143-4bb9-a150-e5aa54a7b1bc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:38.948686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c52c4d-0143-4bb9-a150-e5aa54a7b1bc None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:38.950093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 334/672] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:38.950656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c52c4d-0143-4bb9-a150-e5aa54a7b1bc None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:38.969990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.005798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf74110-53ba-4573-b6ae-61f5c776744b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.043246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.043961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.044316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.044571 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.045408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:39.045703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.047126 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.056565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 339/673] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:39.057374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.081588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c52c4d-0143-4bb9-a150-e5aa54a7b1bc None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.119027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.119722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.120000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.120277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.121045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:39.121358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.122768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e065439-5704-40ae-be39-c7475fb870b7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.136703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 335/674] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:39.138012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.138303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.138585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.159433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:39.160735 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:39.162250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.176076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48665b-ae54-4b8b-9eb9-eebd0371b838 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.213475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.214162 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.214452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.214735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.215810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:39.216147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.218584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.235606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 340/675] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:39.235606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.263095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 336/676] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:39.263095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.265544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.371247 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a2bea4-3608-4c0f-a7cb-7b3a111f07b7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.385247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f133-3ae5-4c4e-b191-7d59b8404639 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.409583 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.410542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.410838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.411122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.424810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.425624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.425977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.426250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.431294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.432839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.434896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.437366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.442652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:39.453574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:39.454787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:39.455913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:39.455913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:39.455913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:39.455913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:39.456168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:39.457375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 337/677] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:39.458473 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.462022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 341/678] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:39.462879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.539838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.581897 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ab4-dd1f-487e-bcbe-20c5f3931ff7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.584579 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256848-9cbf-4968-9557-85fc74d8c446 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.618297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.619128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.619128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.619128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.625033 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.626071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.626460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.626842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.634692 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:39.634904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.636369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:39.647203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:39.648315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:39.649275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:39.650678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 342/679] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:39.651396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:39.652602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:39.653814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:39.655028 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.657871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 338/680] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:39.658773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.725552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.769844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516ea7a-e64b-4925-b8b5-7ab8ea7595aa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.772686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d197f28-2175-405c-a161-e4878d2a6307 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.798413 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.810859 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.811212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.811500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.811780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.812545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:39.812875 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.814496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.816105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.817132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.817605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:39.818066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:39.819066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:39.819561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:39.821316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:39.826784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 339/681] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:39.827674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.842764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 343/682] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:39.842764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:39.959516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec43645-6cc0-4794-aa50-3813731178d6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.960293 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee028b-a7a0-4b61-bce9-10bc22f59c1d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:39.981566 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:39.997939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:39.999672 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:39.999946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.000302 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.001176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.001586 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.003249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.007485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.008243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.008630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.009032 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.009894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.010284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.011781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.018468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 340/683] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:40.018468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.030920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 344/684] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:40.031923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.078313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:40.142303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ede8e8-d52d-41b8-b31b-54d6d9040122 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.149768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b1225-e2b2-4ce2-a23d-5ba281453365 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.180492 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.181306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.181598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.182134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.188233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.188941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.189498 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.189794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.190626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=8b77128461364caebdcab9b93b2aaa5d) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.190797 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.190957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.192241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.192443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.204839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 345/685] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/role_assignments?user.id=8b77128461364caebdcab9b93b2aaa5d&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:40.206240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:40.208904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:40.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:40.211100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:40.215089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 341/686] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:40.216108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.282058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:40.348810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67b19-1564-453c-a894-24943fe1a795 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.362124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bacb-81a1-4157-8008-317a5592b3dd None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.364830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:40.393921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.395107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.395609 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.396095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.397326 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=fac286d2b5514467be990cbea641cb5b) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.397968 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.399666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.402911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.404020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.404967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.405480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.411012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 342/687] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/role_assignments?user.id=fac286d2b5514467be990cbea641cb5b&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:40.412247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.419270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.421398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:40.441294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:40.442346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:40.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:40.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:40.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:40.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:40.443361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 346/688] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:40.448749 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.543403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7fb14-257d-4163-aa14-45db1d106dd2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.578329 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:40.586520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372cba8-6b4a-4b84-88f4-5b8942434632 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.589764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.590832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.591347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.592684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.593806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.594335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.596180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.618652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 343/689] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:40.627142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.637735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.638435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.638759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.639047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.639892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:40.640122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.641573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.656357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 347/690] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:40.662006 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.678202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:40.789124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1ca45-ed51-4c58-b544-1b7a5a438625 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.810232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68297490-ea17-4f5f-baee-d7a3063c02ab None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:40.845300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.846233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.846666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.847141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.863854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:40.864870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:40.865852 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:40.866241 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:40.884237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:40.887055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:40.918622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:40.920067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:40.921364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:40.928983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 348/691] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:40.933324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:40.970750 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.000907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.002413 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.044335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.082440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c455e-6340-4281-af39-e432a4254c11 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.123275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.124778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.125219 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.125668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.126679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:41.127157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.128794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d8929-9abd-49a3-9110-219f05053c28 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.149969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 349/692] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:41.151837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9cd-51e0-472d-bcf6-f9598c14ae44 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.152284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9cd-51e0-472d-bcf6-f9598c14ae44 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.152540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 344/693] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:25:41.153020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a9cd-51e0-472d-bcf6-f9598c14ae44 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.155420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 350/694] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:41.157366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.161084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a9cd-51e0-472d-bcf6-f9598c14ae44 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.273019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0bcd6-82d3-4f19-aa92-5f70b0a01fe0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.284845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.302305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a9cd-51e0-472d-bcf6-f9598c14ae44 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.306665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.316772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.317727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.318199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.318685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.353347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.354288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.354686 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.355072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.369060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.370726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.401102 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-40c11886-ab80-4c2b-a4d2-dc01442c3809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:41.402105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:41.404127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 351/695] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:41.406296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.406888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.406888 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.409629 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.429289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:41.431063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:41.525397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 345/696] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:25:41.525790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 352/697] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:41.527090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.530587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.559112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.612772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.652076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163eb45-f3c0-4f82-ba68-e9ccfb46c730 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.663515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05678d-2de3-4b0a-b970-c6608289fbe2 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.693656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.694494 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.694902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.695248 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.696150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:41.696572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.698110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.705677 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.706786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.707272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.707926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.719207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 346/698] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:41.719207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.727195 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.729056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:41.749754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:41.750776 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:41.751763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:41.751763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:41.751763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.751763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:41.754065 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 353/699] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:41.755176 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.818030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.833591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3696dbe-f08d-477e-a7ef-b438003a1c8f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.881479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f00e9-f24d-4c92-b9c4-8ba59a80c3cc None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:41.889001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.889950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.890337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.890720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.896230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:41.908676 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.910614 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:41.934250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:41.935531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:41.937272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:41.937272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:41.937272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:41.937272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:41.937743 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:41.938778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:41.939484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:41.939976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:41.943669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 347/700] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:41.944739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:41.952940 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:41.954800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:41.975838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:41.977334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:41.978364 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:41.980331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 354/701] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:41.988225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.087481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566f9bb-cb42-4eb5-a6db-42dbe4687dd5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.116562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0b79-0235-4db9-8d55-6e95229cea81 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.124437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.125231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.125523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.125805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.126557 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.126869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.128295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.138812 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:42.144055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 348/702] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:42.145692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.162049 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.163203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.163716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.164203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.165157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.165784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.167508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.187599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 355/703] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:42.188974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.220881 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:42.257886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464a4d4-326c-42e1-bdf4-b2472bda7bd7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.296188 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.296915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.297210 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.297718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.298288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.298604 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.300072 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.308161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc2336c-afff-4ad9-8e9e-c8e977a98e73 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.318410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 349/704] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:42.318410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.355728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.356988 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.357512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.358015 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.366319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.368293 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.380871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:42.385690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:42.386741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:42.387755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:42.391808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 356/705] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:42.393095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.427869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916ace0-392f-43e3-a0b4-1d4613ac5fd2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.465783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.466466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.466755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.467267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.468080 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.469477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.500611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 350/706] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:25:42 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:42.500611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.506927 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:42.535572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c46e3-33ca-431b-8733-39c7fdbb9aac None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.626992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54985-3f91-41c5-8801-a01b1a6355f0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.657423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:42.667423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.668106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.669031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.669308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.671042 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=5df2284d1c2c4e3d81010000420d8646) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.671420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.674182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eea18-94b6-4ad4-afcb-a809cf56951d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.676567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80778682-c977-4de0-af63-31f929522c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.677273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778682-c977-4de0-af63-31f929522c45 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.677576 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778682-c977-4de0-af63-31f929522c45 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.677841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80778682-c977-4de0-af63-31f929522c45 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.678640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80778682-c977-4de0-af63-31f929522c45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.678957 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80778682-c977-4de0-af63-31f929522c45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.680436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80778682-c977-4de0-af63-31f929522c45 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.695295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 351/707] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:42.702155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80778682-c977-4de0-af63-31f929522c45 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:42.703725 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 357/708] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:42 2022] GET /identity/v3/role_assignments?user.id=5df2284d1c2c4e3d81010000420d8646&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:42.837055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80778682-c977-4de0-af63-31f929522c45 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:42.892665 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:42.893400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:42.893689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:42.893978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:42.894836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:42.895153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:42.896729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08728b-8e39-4198-b337-34683492b439 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:42.904125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 352/709] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:42 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:43.004664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fcdca-5f74-4cc0-808c-6c91f84a55d1 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.004985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fcdca-5f74-4cc0-808c-6c91f84a55d1 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.005286 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fcdca-5f74-4cc0-808c-6c91f84a55d1 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.007487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 358/710] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:43.013823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.014630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.014894 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.019960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2690c-06ac-47c0-a749-022019404190 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.020424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2690c-06ac-47c0-a749-022019404190 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.020852 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2690c-06ac-47c0-a749-022019404190 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.022414 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 359/711] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:43.028339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.028692 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.028944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.038073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:43.039312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:43.050135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:43.050912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:43.092319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:43.143598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 353/712] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:43.148471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 360/713] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:25:43.169505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.173665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.285665 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:43.292229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e010f6e-ab9c-46be-a316-7884b1805089 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.315705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f121234-acd8-4b23-bf00-00678d39e633 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.336711 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.337881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.338308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.338696 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.339691 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.342710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:43.362118 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:43.363307 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:43.364258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 354/714] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:43.374138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.377450 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.378696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.379231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.379727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] PATH_INFO: `/v3/services/network` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.380693 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.382315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:43.401605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:43.402787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:43.403735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:43.411600 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:43.416837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 361/715] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services/network => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:43.417808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.487170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85931c2e-38f5-47fb-a84a-6cb431965b2a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.533071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.533771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.534058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.534338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.535082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:43.535398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.537273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.547177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 355/716] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:43.553431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.622545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcb6b0-9f0a-4672-bce7-ff07c3370d0f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.715541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:43.721195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c15ca4-f10e-428e-bf22-5f487dcc33c1 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.733975 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.734685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.734978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.735283 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.736161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] RBAC: Adding query filter params (name=network) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:43.736534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.738130 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.752920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 362/717] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:43.753838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:43.780602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:43.780999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.781701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.781992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.782274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.783009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:43.783324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.784770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b402fd04-d53d-42f4-85f3-8f41c87b559b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.810385 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 356/718] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:43.913298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4dbf1-2cf6-40c6-8758-e863fc587851 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:43.968052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:43.968788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:43.969082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:43.969370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:43.970119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] RBAC: Adding query filter params (type=network) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:43.970443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:43.971866 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751d576-e84f-475a-bd4e-d039d840af2b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:43.978928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 363/719] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:45.611258 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742995-9293-41cc-95bd-4bdd25fd4010 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.611821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742995-9293-41cc-95bd-4bdd25fd4010 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.611821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742995-9293-41cc-95bd-4bdd25fd4010 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.613778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 357/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:45.622210 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d280f-4474-4870-8be1-c0426341fefd None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.622746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d280f-4474-4870-8be1-c0426341fefd None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.622828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d280f-4474-4870-8be1-c0426341fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.647672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-864d280f-4474-4870-8be1-c0426341fefd None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:45.648921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-864d280f-4474-4870-8be1-c0426341fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:45.766232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 364/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:45.802414 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742995-9293-41cc-95bd-4bdd25fd4010 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:45.813287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a53e1d-72d9-4f55-8d2d-4b1e960a22e7 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.813599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a53e1d-72d9-4f55-8d2d-4b1e960a22e7 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.813861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a53e1d-72d9-4f55-8d2d-4b1e960a22e7 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.818482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 365/722] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:45.825953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459698-b846-465e-ac7c-b647831e009e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.827145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459698-b846-465e-ac7c-b647831e009e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.827328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc459698-b846-465e-ac7c-b647831e009e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.838554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:45.859403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-cc459698-b846-465e-ac7c-b647831e009e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:45.859695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-cc459698-b846-465e-ac7c-b647831e009e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:45.947941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742995-9293-41cc-95bd-4bdd25fd4010 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:45.956712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 366/723] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:45.958595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac21be-7e59-4ea6-a742-87ed2367cd2a None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.959132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac21be-7e59-4ea6-a742-87ed2367cd2a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.959741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac21be-7e59-4ea6-a742-87ed2367cd2a None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.961626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 367/724] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:45.977362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.978491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.979220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:45.991688 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:45.992534 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:45.992825 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:45.993108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.009749 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:46.018674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:46.020813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 358/725] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:46.023599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.027447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:46.029069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:46.055496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.120673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 368/726] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 20:25:46.122178 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.139716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63637b23-d611-4faa-82bc-27a4384842b9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.176144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.176755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.177043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.177325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.190511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:46.195517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:46.198372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:46.199657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:46.200648 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 359/727] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:46.206034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.246135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97827e6-a020-46e1-9b99-de2c2ac5fb06 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.287130 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.288373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.289054 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.289582 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.290622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:46.291877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.293577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.308181 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 369/728] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:46.308910 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.310630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.329486 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b342b-f76f-458a-83ca-30fd9674e691 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.367914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.368756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.369042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.369319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.383584 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:46.392320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:46.394164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 360/729] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:46.395670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.475697 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebad489-41c3-43f1-8f66-275abf60386d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.513592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.522608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896b9f-0e82-455d-91b9-e42339bc2b83 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.525147 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.525950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.527084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.527084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.527281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:46.527585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.529031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.544270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 370/730] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:46.545363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.574783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.576945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.577476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.578351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.593801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.595143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.604829 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:46.616213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:46.617817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:46.618834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:46.618834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:46.618834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:46.618834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:46.624740 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 361/731] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:46.625933 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.697124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc067dd-0779-4e93-953e-ad856d27b371 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.743287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.744348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.744679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.745117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.745902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:46.746220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.747616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.758211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef75c-9c07-445d-bafa-5724efa52193 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.758742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 371/732] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:46.759494 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.797930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.798735 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.799023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.799304 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.810413 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.812383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.816841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d14f7dec-d4ed-403a-adca-4d87a2ca315a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:46.831212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:46.832462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:46.833653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:46.833653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:46.833653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:46.833653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:46.836549 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 362/733] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:46.837892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eb0a-6a13-4bb2-841b-9051a346cc80 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.838185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eb0a-6a13-4bb2-841b-9051a346cc80 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.838518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7eb0a-6a13-4bb2-841b-9051a346cc80 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.839838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 363/734] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:46.840578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7eb0a-6a13-4bb2-841b-9051a346cc80 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.877339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63186ab1-42ef-4dea-beb2-f9760eeef71c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.893745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:46.913252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.913934 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.914221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.914600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.915240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:46.915558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:46.917107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959aefa3-f8d1-48b4-b088-352a6581f153 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:46.930755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 372/735] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:46.932107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9635-fa0d-46b1-8e49-33ae1be93f9f None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.932413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9635-fa0d-46b1-8e49-33ae1be93f9f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:46.932705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9635-fa0d-46b1-8e49-33ae1be93f9f None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:46.933936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 373/736] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:46.934631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9635-fa0d-46b1-8e49-33ae1be93f9f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:46.962606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7eb0a-6a13-4bb2-841b-9051a346cc80 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:46.998867 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:46.999577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:46.999860 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.000139 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.010817 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.012383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-939aa6e7-d40e-477e-8b4a-cddea93bc2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:47.037354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:47.038444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:47.039428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:47.039428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:47.039428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:47.039428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:47.040190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 364/737] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:47.041855 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c544-2439-4cc2-839c-9693ec4bf177 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.042151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c544-2439-4cc2-839c-9693ec4bf177 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.042437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c544-2439-4cc2-839c-9693ec4bf177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.051680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d9635-fa0d-46b1-8e49-33ae1be93f9f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.063447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.064975 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-d521c544-2439-4cc2-839c-9693ec4bf177 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:47.066253 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-d521c544-2439-4cc2-839c-9693ec4bf177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:47.091694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.092420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.092719 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.092992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.093754 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:47.094029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.095409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.106876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 374/738] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:47.108502 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.143345 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 365/739] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:47.144773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.145077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.145364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.231922 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ccbc4-081e-44e3-8f6d-a001b36beb4d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.272435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.273177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.273467 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.273755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.291504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.293019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.302565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:47.303794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:47.308814 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:47.309989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 375/740] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:47.311128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.330074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.353341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.379148 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 366/741] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:25:47.380056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.441003 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d84829-2ec9-495e-997f-9119470ee507 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.477002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.477689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.477979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.478262 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.479000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:47.479319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.480737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.493710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 376/742] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:47.494563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.499257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829de3f-92e6-4db0-a383-349a7db30159 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.535884 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.536695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.536984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.537263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.546101 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.547590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:47.563055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:47.564167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 367/743] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:47.565988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.586174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.607189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.617846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc7aa2-37ba-4f7b-9e86-9d34284f3b88 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.653874 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.654642 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.654974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.655316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.656071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.657510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.673356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 377/744] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:25:47 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:47.674174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.685439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76357c15-64e3-452d-9ba2-fea122327827 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.737285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.738182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.738477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.738762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.739653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:47.739974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.741547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.762623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 368/745] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:47.763599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.822599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122780b0-e5ae-4ba5-a8c5-12ed4d86950b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.862429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.875072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.875518 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.875884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.876240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.877106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:47.878575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:47.893594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 378/746] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:25:47 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:47.894444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:47.905470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef8c8f-5997-4e9d-8fd0-38d97ca78842 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:47.914390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:47.982040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:47.984206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:47.985266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:47.986057 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:47.997871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.000233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:48.020777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:48.021956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:25:48.023010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:48.026226 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 369/747] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:48.027487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.037621 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881638e-199f-40b6-acc6-cc3ca00257b8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.086361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.086986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.087277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.087558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.088846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:48.089178 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.091067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.118448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 379/748] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:48.119770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.166185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01d2fb-f2be-458c-a3fe-c3f11a52ac90 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.204232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.207524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.216390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.218142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.218730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.219300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.252760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b3afab-901e-428b-9386-953b3d5c05de None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.299804 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.301320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.301622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.301915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.302780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:48.303176 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.305495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.317180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.319365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.336559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 380/749] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:48.337857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.458090 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 370/750] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:25:48.459573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.469033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68105abd-31e3-4993-a95f-6e46f9f1a36d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.492703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.506370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.517945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.519156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.519716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.520309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.532958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.534977 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:48.556192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:48.557313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:48.558357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:48.558357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:48.558357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:48.558357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:48.561537 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 381/751] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:48.562573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.578597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec18d4e-7c21-4e18-8f11-f556d03e527b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.619053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.619635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.619920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.620200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.677782 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eda1a-131e-49ab-989a-fd2b0d4966d9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.700359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.701915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.719561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.721021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.721738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.722608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.734322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.740935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.743475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:48.767729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:48.768886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:48.769884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:48.769884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:48.769884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:48.769884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:48.772994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 382/752] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:48.774058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.816124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:48.838689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 371/753] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:25:48.848545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.913492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640342f-2464-42e9-98be-b2121d411dd5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:48.952684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:48.953482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:48.953740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:48.954027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:48.954772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:48.955146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:48.956480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:48.970351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 383/754] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:48.971224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:48.979870 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5717ad5-597f-42d2-9b42-041ab466dc99 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.011728 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.022743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.023797 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.024460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.025077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.026022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:49.026543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.028354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.047630 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 372/755] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:49.049228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.074288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.085835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72842a-9343-4f27-b9ac-5b908eab17a7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.126873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.127682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.127970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.128248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.140605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.142170 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:49.163198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:49.164380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:49.165434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:49.165434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:49.165434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:49.165434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:49.168877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 384/756] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:49.170043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.176602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9f3e-2e08-4e24-9bae-438420d5ed61 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.217572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.218765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.219366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.219940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.228271 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.230213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:49.247320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:49.249061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.250089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:49.253140 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 373/757] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:49.254340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.285287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91799ff-5da8-422c-8a30-cc0a0481d235 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.315932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.331289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.332105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.332417 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.332729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.343208 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.345648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:49.365863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:49.366938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 385/758] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:49.373993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.375656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.411524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a105b0-1d82-4608-be62-c02f064440dd None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.489474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.490590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.491073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.491562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.492902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:49.493436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.496247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.526578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 374/759] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:49.527596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.559053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739816c3-6355-4fae-860d-1b55119aac70 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.598158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.598865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.599118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.599397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.600206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:49.600549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.602015 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.615953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 386/760] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:49.617075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.670988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dbe1-4ef3-415f-93cd-3fd1524ba18f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.682593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.699343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:49.723571 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.725790 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.726375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.726838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.728137 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:49.729164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.731041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.746849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ed1a-0f86-4661-bf96-e88688848a5b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.754591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 375/761] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:49.754591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.793309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.794547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.794742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.795365 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.802672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.804312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:49.821012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:49.822424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:49.823787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:49.830611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 387/762] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:49.830611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:49.918061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a894e-c015-466c-899b-b5bc8a362c2b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.962607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9169c-1e34-43c7-b7ff-87380b57962f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:49.974768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:49.976260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:49.976916 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:49.977421 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:49.979773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=d4759f19e262476a8c4c4a99b43daa13, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:49.981599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:49.992452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:50.002754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 376/763] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:49 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:50.004127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.018906 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:50.020348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.020882 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.021508 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.023535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=5df2284d1c2c4e3d81010000420d8646) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:50.024130 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:50.026482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e292f7-5615-42ab-9585-55f88c7956e1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:50.040521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 388/764] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:49 2022] GET /identity/v3/role_assignments?user.id=5df2284d1c2c4e3d81010000420d8646&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:50.048906 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:50.098721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb7172-3cfc-4e1c-b428-33d9b77ed82d None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.099081 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb7172-3cfc-4e1c-b428-33d9b77ed82d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.099299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb7172-3cfc-4e1c-b428-33d9b77ed82d None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.101339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 389/765] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:50.112307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.112709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.112914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.147925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:50.149757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:50.198477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6901d-9a8e-48a0-98bb-b5dab475b2cf None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:50.237807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:50.238373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.238666 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.238952 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.239840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:50.240165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:50.241750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:50.258219 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 377/766] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:50.264821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.294193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 390/767] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:25:50.333334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.387758 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:50.404833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f1aa-f8f1-4931-a6b0-b60725e29946 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:50.456170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:50.456930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.457221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.457510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.459025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=b8ab51b1618f461bb8918f85b58c2793) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:50.459351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:50.461644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793b20c-0286-4a39-969a-b4794c8121a6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:50.465935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbb0e7-8842-4c00-83b3-74e3a7ec9b96 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:50.474993 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 378/768] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/role_assignments?user.id=b8ab51b1618f461bb8918f85b58c2793&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:50.476573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e363ca8-56d5-4835-9a35-bb62403a77bf None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.476872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e363ca8-56d5-4835-9a35-bb62403a77bf None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.477159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e363ca8-56d5-4835-9a35-bb62403a77bf None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.480487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 379/769] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:50.485879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.486502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.486980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.493506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:50.519871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:50.521036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:50.533095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:50.533934 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.534221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.534501 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.535183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:50.536616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:50.565484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:50.566710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:50.568300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 391/770] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:50.573153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.653283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 380/771] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:50.678003 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.807023 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6c143-ef0a-4973-a026-73ae131f09ed None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:50.845759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:50.869532 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:50.870770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:50.871310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:50.871820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:50.873422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:50.874086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:50.875774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:50.881682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 392/772] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:50.887230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:50.894629 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832bd38-e231-4060-b60f-1cdfb75d6d3a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.006209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.007047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.007335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.007612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] PATH_INFO: `/v3/services/network` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.008294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.009747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:51.034387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:51.035606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:51.037094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:51.048191 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 381/773] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/services/network => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:25:51.054626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:51.168785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:51.210094 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34708c09-b271-4ab1-ba82-d5d2ad9eeca8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.251485 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.252599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.253133 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.253635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.254589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:51.255296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.256949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.263690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 393/774] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:51.272854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:51.378409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbbf2a-233e-4ad7-b221-2666043908da None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.425987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886a5a5-4236-4d62-9cbc-67a3277dc399 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.447299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:51.453218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.455285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.456040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.456842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.457633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] RBAC: Adding query filter params (name=network) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:51.457955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.459417 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.483366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 382/775] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:51.484239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.485265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.485777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.486280 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.487292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] RBAC: Adding query filter params (interface=public, service_id=70f42b791382486a8be312bc54f9113e, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:51.487921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.489063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:51.489800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.503842 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 394/776] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:25:51 2022] GET /identity/v3/endpoints?service_id=70f42b791382486a8be312bc54f9113e&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:51.510221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:51.518458 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:51.676520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d3526-aa43-4761-a756-40b73218b2d2 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.731771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167a1b7-3118-4801-a303-3df29195ec65 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:51.752309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.753361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.755008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.756127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.757513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.760707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.771069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 395/777] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:25:51 2022] GET /identity/v3/services => generated 772 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:51.795355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:51.796583 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:51.797146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:51.797651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:51.798628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] RBAC: Adding query filter params (type=network) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:51.799183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:51.800911 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:51.808681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 383/778] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:51.816149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:51.869622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:51.975425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c1ef8-6148-477f-a6ab-13317eedf0d9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:52.027237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:52.028343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:52.028899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:52.029507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:52.030615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] RBAC: Adding query filter params (interface=public, service_id=5af83c1feb3a4860a088ad1a41fa3a2b, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:52.031178 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:52.032873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da2713-8e0e-492e-9718-24c32a403489 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:52.046024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 396/779] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:25:51 2022] GET /identity/v3/endpoints?service_id=5af83c1feb3a4860a088ad1a41fa3a2b&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:52.052007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:52.170809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:52.212325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26731f-3079-43f9-888b-31609e0763f7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:52.257625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:52.258221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:52.258516 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:52.258788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:52.259488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:52.260903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13782-9146-46fc-b0da-ce194a96bc48 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:52.267700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 384/780] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:25:52 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:52.703777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab7a-9e01-40b7-bc0b-4f6aa46f0074 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:52.704347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab7a-9e01-40b7-bc0b-4f6aa46f0074 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:52.704347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab7a-9e01-40b7-bc0b-4f6aa46f0074 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:52.706129 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 397/781] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:52.711551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b2ba-19b7-4a39-9f81-6c02c86851cc None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:52.711862 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b2ba-19b7-4a39-9f81-6c02c86851cc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:52.712129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b2ba-19b7-4a39-9f81-6c02c86851cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:52.733332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-31f9b2ba-19b7-4a39-9f81-6c02c86851cc None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:52.734657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-31f9b2ba-19b7-4a39-9f81-6c02c86851cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:52.826639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 385/782] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:52.876490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cab7a-9e01-40b7-bc0b-4f6aa46f0074 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.025604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:53.029579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cab7a-9e01-40b7-bc0b-4f6aa46f0074 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.073482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61cce-45c7-47bc-a52d-5dcfd8c2cbbf None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.074468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61cce-45c7-47bc-a52d-5dcfd8c2cbbf None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.075012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61cce-45c7-47bc-a52d-5dcfd8c2cbbf None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.077540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 386/783] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:53.081640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.081640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.081640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.081640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.083562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.084085 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.084643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.096213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:53.104091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:53.105292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:53.106250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 398/784] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:53.108518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:53.110316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:53.113865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.233858 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:53.245079 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 387/785] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:53.271308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.284428 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbd885-dd70-45e7-845c-0edc2228c623 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.344788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.346185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.346975 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.347491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.349214 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:53.349745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:53.352311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be80c01-cbef-44ce-a655-d568b4e851ff None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:53.365755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 399/786] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:53.367521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96197-bac4-4729-8a7e-dda485e86303 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.368058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96197-bac4-4729-8a7e-dda485e86303 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.369352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f96197-bac4-4729-8a7e-dda485e86303 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.371925 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 400/787] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:53.373794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96197-bac4-4729-8a7e-dda485e86303 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.408078 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:53.410060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d7e8b-c23c-462b-8d2b-0f76cbc072bb None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.463397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.464212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.464523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.464802 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.479744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-eb54ee7d-6045-4129-977a-8a9a1b5f5e57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:53.490554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:53.491686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:53.492656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 388/788] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:53.493462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.494064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.494064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.516948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:53.518320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:53.531462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f96197-bac4-4729-8a7e-dda485e86303 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.577729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.579542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.580064 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.580628 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.596671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:53.597565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:53.599840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:53.620670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 389/789] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:53.621589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:53.622588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:53.624312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:53.624312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:53.624312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:53.624312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:53.625211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.626982 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 401/790] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:53.633769 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.738989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:53.777774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35065bd-c896-4c7e-ad30-20107a9f7c4b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.780791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47854a7-1a51-43db-932a-2bcfe689ea88 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:53.832189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.832965 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.833450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.833945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.834827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:53.835242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:53.835509 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:53.836440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:53.836880 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:53.837114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:53.837493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:53.838389 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:53.848220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:53.849203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 390/791] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:53.850180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.850474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:53.866192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 402/792] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:53.867488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:53.923780 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.003518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b08d5a-8c2b-4a08-a5fd-61728a1b1b9f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.018607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8190ac-3a97-4cfa-8e5e-44977ea701d8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.062886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.063878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.064170 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.064740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.067468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.078008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.079031 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.079757 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.081839 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.082897 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.083692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:54.100575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:54.102495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 403/793] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:54.105206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.117099 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:25:54.126719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:54.128667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:25:54.130353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:54.132911 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 391/794] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:25:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:54.133804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.259644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5fd60-b75d-40de-9c2b-055ccf78b476 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.274472 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.289774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837dd6c-885f-458a-a0ff-994a101f544f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.307987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.308734 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.309025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.309311 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.310071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:54.311096 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.312884 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.329872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 392/795] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:54.330936 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.338224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.339366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.339871 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.340367 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.349504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.351468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:54.368645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:54.369836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:25:54.370913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:54.374149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 404/796] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:54.375246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.447965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.464499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab9b83-16e9-4d11-84bb-cb57d70563ac None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.516875 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.517578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.517866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.518150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.519243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:54.519568 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.521014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.523121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6e166-4b38-4d24-833b-de99c4f13c5d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.536739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 393/797] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:54.538010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.571623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.572669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.572829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.573111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.583750 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.584809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.586920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:54.605213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:54.606306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 405/798] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:54.608100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.669642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2d48-04dc-4c93-b9a6-e61d39fb33ba None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.710717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.711411 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.711707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.711986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.713342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:54.713663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.715289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca62aab-13ea-42cb-bae5-24ddf832282b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.726413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69127ec3-3aa7-4fb3-9b04-539bd5ffe0ea None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.735849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.737114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 394/799] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:54.743078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa35be-8a8b-49e8-9661-0ee3e4b0c3f4 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.743371 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa35be-8a8b-49e8-9661-0ee3e4b0c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.743657 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa35be-8a8b-49e8-9661-0ee3e4b0c3f4 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.745020 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 395/800] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:54.748564 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa35be-8a8b-49e8-9661-0ee3e4b0c3f4 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.771011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.771970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.772262 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.772889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.788102 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.789592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:54.811172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:54.812270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:54.813634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:54.813634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:54.813634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:54.813634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:54.813634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 406/801] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:54.814967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:54.866322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:54.878885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa35be-8a8b-49e8-9661-0ee3e4b0c3f4 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.922273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.922968 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.923252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.923531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.924435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:54.924758 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:54.926279 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c7bc9b-66bc-4c53-ae80-f0adc9121e97 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:54.943648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b3cb7-5205-43b1-90ec-77c9761e620d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:54.952970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 396/802] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:54.952970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.952970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.952970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:54.971348 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:54.972683 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:54.986334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:54.987067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:54.987361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:54.987753 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.020275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.058276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 397/803] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:55.058276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.075008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.077511 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.156972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.180442 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d916a-1677-4c01-bad6-0995a7bcba73 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.188622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 407/804] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:25:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:25:55.189739 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.224509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.225180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.225467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.225760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.226497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:55.226815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.228203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.240154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 398/805] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:55.241094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.253876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.298631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f91f59-1d11-457e-ae52-ec592f8d0259 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.342764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.343575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.343860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.344146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.355085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.356637 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.364466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf926c-bf65-4934-bc36-28e1472203a4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-8f429069-82c2-419d-b9a5-39c9d84ecf53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:55.376708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:55.377714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:55.378748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:55.378748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:55.378748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:55.378748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:55.378748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 408/806] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:55.379332 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35237670-cdc5-4222-b66e-dfd82e9089c5 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.379610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35237670-cdc5-4222-b66e-dfd82e9089c5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.379875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35237670-cdc5-4222-b66e-dfd82e9089c5 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.381184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 409/807] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:55.381892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35237670-cdc5-4222-b66e-dfd82e9089c5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.405743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.406937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.407408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.407789 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.408808 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.411297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:55.425376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:55.431771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:55.433177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 399/808] 10.222.0.13 () {56 vars in 1129 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:55.434362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.434362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.498593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35237670-cdc5-4222-b66e-dfd82e9089c5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.510106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.536559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.537367 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.537657 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.537940 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.538491 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1780-f894-4e4a-89ea-8e412d520688 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.548711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.550164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:55.569334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:55.570398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 410/809] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:55.571385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.575930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.577712 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.578006 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.578289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.586423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.587896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-9c43a81c-24d8-4653-81cf-e14d64fb9404 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:25:55.603693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:55.605041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:55.606115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 400/810] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:55.606944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.606944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.606944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.625485 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:55.626707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:55.686916 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.693520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab3c8-152d-47a0-aebb-14c841f94941 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.698465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 401/811] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:55.699308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.744076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.744777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.745079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.745366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.746109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:55.746437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.747851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.759459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 411/812] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:55.760303 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.789883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.807763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017d9c3-e9e7-4e74-98e0-a3cf9ec629aa None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.845422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.846097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.846388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.846664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.847376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:55.847687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.849121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.853915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 402/813] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:55.854687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.881861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebc144-c6b1-4908-8f92-aee4a0bf600f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:55.917684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:55.918213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:55.918901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:55.919192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:55.919486 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:55.921034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:55.921404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:55.922737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:55.937091 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 412/814] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:55.937880 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:55.969309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085520d-8a5a-40cb-8b8a-5d94d9ef13a5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.009929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.010651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.010946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.011233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.012120 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.012445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.014496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.025449 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.033616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 403/815] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.034541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.047848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b5dd-0e6c-4f38-92e3-07c7b59bcd09 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.085568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.086403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.086689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.086969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] PATH_INFO: `/v3/services/network` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.087660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.089099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:56.102408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:56.103635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:56.105419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 413/816] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:55 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:56.106416 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.151324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166f875-ed88-4ab7-b1f2-61f04ab84bd5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.167874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.198841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.199572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.199865 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.200152 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.208801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.210273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.216087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac2c-807c-4e4c-b706-2729d8409148 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:56.224252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:56.225466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:56.226498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 404/817] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:56.227050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.254862 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.255534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.255819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.256098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.256848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.257223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.258527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.264591 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 414/818] 10.222.0.13 () {56 vars in 1124 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.265258 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.281677 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.346539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a738fa-27a6-4da0-b1d3-b6a62dec785f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.382687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.382949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee4f6a-cd89-458c-baf5-597a7fa00a71 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.383499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.383786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.384066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.391892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.393428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:56.409045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:56.410305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 405/819] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:56.411513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.425412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.426214 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.426529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.426808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.438488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.440198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.448480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:25:56.468185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:56.473965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 415/820] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:56.475223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.528413 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba89b0-1b51-4112-b1af-08ff80dc84b6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.539481 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.568915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.569587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.569873 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.570151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.570868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] RBAC: Adding query filter params (name=network) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.571178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.572605 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.577833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 406/821] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.578603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.587803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba8652f-8a83-42f2-9a9f-d1397cb9f3ad None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.625323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.625994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.626277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.626553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.627347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.627663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.629134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.641860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 416/822] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.642865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.687474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.694625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c47d9-75b6-47c7-9bd4-5ab17475e4a7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.735306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.735884 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.736176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.736479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.737230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.738620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.762606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 407/823] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:25:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:56.762606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.778206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7e80e-a00b-4eef-afe5-48169ac639a4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.808470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:56.839866 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.840580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.840871 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.841213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.842174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.842885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.844234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.868679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 417/824] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.868679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:56.923270 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4872d4-7028-4f95-9c6a-38488e249b6f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:56.965497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:56.966204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:56.967066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:56.967363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:56.968155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:56.968513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:56.969995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:56.989498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 408/825] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:56.989498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.003442 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:57.009306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ddd1b-7c1a-4b33-8a74-c5a593e0fa5d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.052808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.053493 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.053783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.054063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.054777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] RBAC: Adding query filter params (type=network) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:57.055096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.056496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.066704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 418/826] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:25:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:57.066704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.113208 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:57.115101 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdb354-73f2-4204-bf83-39b09832ad66 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.167433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.168361 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.168934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.168934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/5df2284d1c2c4e3d81010000420d8646/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.171941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=5df2284d1c2c4e3d81010000420d8646, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.173510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.191685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb34678-bfc2-4870-b20d-c1fd9c104479 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.195515 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 409/827] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:56 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/5df2284d1c2c4e3d81010000420d8646/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:57.196215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.247111 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.248474 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.249242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.249819 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.250985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:57.251719 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.253717 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.269252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 419/828] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:25:57 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:57.270728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.304539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:57.345389 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141316b-343c-4ff6-abb5-3321483df84b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.401323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.402250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.402551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.402840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.416346 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.417730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.436443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293a075-9166-406e-bba1-7e06873e194f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.446651 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:25:57.455237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:57.456286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:25:57.457313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:57.462485 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 410/829] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:57.469372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.498799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.499918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.500689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.501255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.502437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.503958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.534625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 420/830] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:25:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:25:57.663498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f4a02-da34-4616-8d52-1565de5187d5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.722108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:57.727691 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.728604 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.728904 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.729610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.730460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:57.730786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.733670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f8e9c-7eda-4c37-8ac5-d462da604c14 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.750432 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 411/831] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:25:57.756961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:57.888989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efefd0-3bed-4c39-a4a0-a50837477952 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:57.937084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:57.938016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:57.940082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:57.940379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/b8ab51b1618f461bb8918f85b58c2793/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:57.958551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=b8ab51b1618f461bb8918f85b58c2793, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:57.960082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9949-046c-4d8c-a050-cf56f86e31a1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:57.981346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 421/832] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:25:57 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/b8ab51b1618f461bb8918f85b58c2793/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:25:59.377547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef497-0ede-45a4-a33d-c29f2f983cb5 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.378087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef497-0ede-45a4-a33d-c29f2f983cb5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.378087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ef497-0ede-45a4-a33d-c29f2f983cb5 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.379771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 412/833] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:25:59.385305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.385768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.385861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.404773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:59.406051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:59.414104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:59.486258 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 422/834] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:25:59.508959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ef497-0ede-45a4-a33d-c29f2f983cb5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:59.630710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ef497-0ede-45a4-a33d-c29f2f983cb5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:59.668848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:59.670165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.670457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.670742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.671582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:59.673020 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:25:59.696518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:25:59.697713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:25:59.698655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 413/835] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:25:59.705405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:59.824515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa959c5b-2f60-430b-ba7e-218dabaec89d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:25:59.869055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:25:59.869774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.870114 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.870403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.871152 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:25:59.871473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:25:59.873705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5a249-6a11-4d08-8b4c-c9546a5ccf59 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:25:59.879816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 423/836] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:25:59.885926 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:25:59.919134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2be0c-1810-4324-b126-42349fcf4802 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.919424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2be0c-1810-4324-b126-42349fcf4802 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.920590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2be0c-1810-4324-b126-42349fcf4802 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.922963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 424/837] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:25:59.928373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:25:59.928696 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:25:59.929011 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:25:59.941894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:25:59.949978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:25:59.952071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:25:59.968881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.021678 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07c81f-823b-402b-ae33-a720cb1bb54d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.032700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 425/838] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:25:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:00.052537 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.068388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.069856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.070662 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.071176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.072291 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:00.072799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.074382 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febfa08-d9f0-40e5-a327-64d53fe575ae None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.079750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 414/839] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:25:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:00.081298 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b018-9c5f-480a-9dd4-6474753943db None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.081595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b018-9c5f-480a-9dd4-6474753943db None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.081886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969b018-9c5f-480a-9dd4-6474753943db None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.083419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 415/840] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:00.089106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.089463 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.089674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.112895 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:00.114524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:00.185635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e7c80-32c8-4107-8e75-2075d8dd2d34 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.228834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 416/841] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:00.232535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.233694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.234220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.234716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.238296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.244052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.254761 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:00.262836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:00.264979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:00.266010 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:00.270217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 426/842] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:00.271067 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.325568 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.392313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e7e53-1d0e-4298-941f-aaf05459bd11 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.421496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674f473-916c-43c8-82e6-7b341f4592d4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.428839 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.429671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.429961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.430244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.430909 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.432383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-790bf0c8-d36e-4680-a2b8-a47e879ceffe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:00.445323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:00.446465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:00.447487 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 417/843] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:00.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c79d8-3978-446f-9d8c-bbf4e40dd581 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c79d8-3978-446f-9d8c-bbf4e40dd581 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.448543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116c79d8-3978-446f-9d8c-bbf4e40dd581 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.449762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 418/844] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:00.453283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c79d8-3978-446f-9d8c-bbf4e40dd581 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.469936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.470626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.470916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.471203 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.471979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:00.472296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.473721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e4a8b-978b-417e-9d68-79fa1812d78c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.482782 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 427/845] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:00.483833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.484131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.484447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.503645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:00.505269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:00.574556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.589230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116c79d8-3978-446f-9d8c-bbf4e40dd581 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.593608 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 428/846] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:00.601222 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.604590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.630032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.630778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.631070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.631354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.632087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:00.632496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.633916 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.640596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 419/847] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:00.641805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.741680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd77e39-e6ad-4344-aae0-efdc6effa4d7 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.767956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b76625-a925-4a89-9035-c117d8e18e59 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.784732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.785548 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.785856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.786109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.797379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.798763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.807364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.808178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.808498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.808786 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:00.818171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:00.819378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:00.820635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:00.820635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:00.820635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:00.820635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:00.820635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 429/848] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:00.820971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.825419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:00.834032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:00.835262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 420/849] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:00.836283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:00.878601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.890522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:00.955273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039e95e-4fcc-4534-b2fb-8e69931549b7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.958916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fdb21-1303-46ac-9a1b-98f6d3250bec None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:00.995320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.995886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.996222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.996565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.996921 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:00.997316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:00.997602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:00.997755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:00.998020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:00.998270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:00.998961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:00.999084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:00.999271 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.000716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.005706 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 430/850] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:01.006437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.041051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 421/851] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:01.046928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.139385 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233de6be-5849-47a9-8252-5885e9fdaef0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.143315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.185051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.186470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.187371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.187560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.188875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:01.189264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.191116 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 431/852] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:01.209401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.228418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.246207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973de00-ef9d-445b-82cd-bdd3407c9f43 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.289529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.290388 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.290677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.290957 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.300192 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.301733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:01.318643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:01.319877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:26:01.321089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:01.324754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 422/853] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:01.326739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.333302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12195a4-b370-4666-a500-880dd0f91f8f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.370445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.371261 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.371551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.371836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.384587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.386186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:01.405716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:01.406875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:01.407970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:01.407970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:01.407970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:01.407970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:01.407970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 432/854] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:01.413780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.435953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.457134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4deb4-594d-483e-9f3c-48840b7ca4f7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.498190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.499675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.500255 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.500824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.501966 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:01.502543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.504585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.527526 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 423/855] 10.222.0.13 () {56 vars in 1135 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:01.539805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.542394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06fc7d-a5bc-44e1-97b3-b0d7f4db6e30 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.589092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.589618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.590946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.591243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.592329 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.593992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:01.594326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.596298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.615369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 433/856] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:01.621068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.675974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c10ca-fbc0-4b00-a3cc-4f07754faaf9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.722722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.726406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.727663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.728179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.728900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.751307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713ecb8-6699-487f-884a-43192d35894e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.800743 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:01.801905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:01.803058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:01.803564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:01.815401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.815944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:01.816974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.818449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:01.839719 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:01.839993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:01.842101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:01.844251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:01.849243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 434/857] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:01.850308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:01.976084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe8e93-fb31-4470-8ba8-5117816e158d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:01.977460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 424/858] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:01.987346 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.016034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.016860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.017255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.017635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.018709 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:02.019142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.020699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.031823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.048984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 435/859] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:02.049931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.131327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a570cb2-121d-4b5e-9d4b-62e955ede3e4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.154228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.173558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee239df-1f86-4b76-98b6-671e8078282e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.182722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.183543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.183829 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.184106 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.199943 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.202212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:02.225794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:02.227205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:02.228484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:02.228484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:02.228484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:02.228484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:02.228484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 425/860] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:02.230407 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.231347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.231939 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.232780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.232929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.315348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.325008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.327747 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.356338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59974416-9049-4982-b9ff-1ee671552619 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.395410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.396084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.396306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.396626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.397390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:02.397703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.399288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.417206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 426/861] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:02.418189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.445635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 436/862] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:02.453026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.456475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.537396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed04cfd-bf58-4a2a-a17e-dd5226bca9ed None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.570427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.577961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a20ed-51e4-41c4-9ef8-4121eb6fbf60 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.588642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.589871 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.590358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.590597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.599602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.601261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.616026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.616861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.617154 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.617431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-35b07235-815c-4083-82c9-f6dd8dc648fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:02.618826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:02.619868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 427/863] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c07616-28f1-49f4-9309-30e9b2b2f859 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.626821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c07616-28f1-49f4-9309-30e9b2b2f859 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.627567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c07616-28f1-49f4-9309-30e9b2b2f859 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.627567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 428/864] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:02.628677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c07616-28f1-49f4-9309-30e9b2b2f859 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.637127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.638827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-51b7bd86-f464-4771-9df5-c89571c5ae61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:02.657601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:02.658774 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:02.659912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:02.659912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:02.659912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:02.659912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:02.659912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 437/865] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:02.660372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.661499 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.661499 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.678778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:02.680009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:02.736739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.756627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c07616-28f1-49f4-9309-30e9b2b2f859 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.764603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 438/866] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:02.765955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.796545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.797224 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.797639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.797826 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.798629 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:02.798954 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.800446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.814225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 429/867] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:02.815320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.876175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:02.903246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008929-bdb6-4db1-9b26-4800664e8f4b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.932535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3af8b4-f08e-4adb-a6b2-e600d0b02174 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:02.943973 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.944797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.945093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.945405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.946119 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:02.946439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.947822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:02.960864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 439/868] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:02.965965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:02.970925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:02.971701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:02.972170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:02.972506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:02.973274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:02.974768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.000434 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 430/869] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:02 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:03.000434 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:03.011836 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:03.116186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d28b-08c6-4940-95ce-18286b54a9ef None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:03.166193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:03.166946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.167234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.167515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.168104 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003a834-aa9b-4b36-9d14-336092295611 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:03.168467 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=5df2284d1c2c4e3d81010000420d8646) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:03.168793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:03.171439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.174001 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:03.180886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 440/870] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/role_assignments?user.id=5df2284d1c2c4e3d81010000420d8646&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:03.226055 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:03.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.227200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.241667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:03.242624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:03.259813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:03.262189 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:03.265229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 431/871] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:03.268760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:03.402113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffec7e-874a-4f03-a284-82095f54fcca None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:03.444343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:03.445086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.445376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.445658 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.446473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:03.447022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:03.448244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ce69a-7555-4c32-bc7b-00582e2edbf6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.463822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 441/872] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:03.470680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:03.495659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6337a2-a9a5-4d0a-9e85-dd38c35ca572 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.495659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6337a2-a9a5-4d0a-9e85-dd38c35ca572 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.495659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6337a2-a9a5-4d0a-9e85-dd38c35ca572 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.495659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 442/873] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:03.497277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.497571 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.498504 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.522829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:03.525238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:03.549425 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:03.610153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c648417-d64d-403d-81d1-21c886c340d2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:03.618079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:03.636274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 443/874] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:03 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:03.657172 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:03.657784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.658077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.658364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.659250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=b8ab51b1618f461bb8918f85b58c2793) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:03.659575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:03.661178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37ff5b-42db-42ea-93a5-2eed351c62a0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.662343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:03.669352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 432/875] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:03 2022] GET /identity/v3/role_assignments?user.id=b8ab51b1618f461bb8918f85b58c2793&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:03.790820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4a514-c450-4233-a888-be9fedc119e6 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:03.833425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:03.834388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:03.834864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:03.835309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:03.836218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:03.837851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bd624-77c5-4594-ab77-8aac64163b5e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:03.868561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 444/876] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:03 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:26:05.409543 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd12a-bcca-4a00-ae41-26d96a0e5402 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.410135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd12a-bcca-4a00-ae41-26d96a0e5402 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.410135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd12a-bcca-4a00-ae41-26d96a0e5402 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.411999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 433/877] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:05.418285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f09a-56a4-42e0-8f87-e4a211d87620 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.418865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f09a-56a4-42e0-8f87-e4a211d87620 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.418865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7f09a-56a4-42e0-8f87-e4a211d87620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.438230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-88e7f09a-56a4-42e0-8f87-e4a211d87620 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:05.439695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-88e7f09a-56a4-42e0-8f87-e4a211d87620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:05.489792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:05.522705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 445/878] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:05.545582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd12a-bcca-4a00-ae41-26d96a0e5402 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:05.624541 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a05abb-e676-4a50-8de0-3b9c606a56ac None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.625224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a05abb-e676-4a50-8de0-3b9c606a56ac None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.625795 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a05abb-e676-4a50-8de0-3b9c606a56ac None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.628777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 446/879] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:26:05.637143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.637594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.638033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.658560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:05.659970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:05.675234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd12a-bcca-4a00-ae41-26d96a0e5402 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:05.742010 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:05.750965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 447/880] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:05.754354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:05.755358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.755807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.756589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.757465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:05.759020 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:05.769205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:05.773405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:05.774835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:05.776157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:05.784433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 434/881] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:05.786524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:05.886680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce55fa-2000-4bc7-952b-e672dcc3a10a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:05.890058 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:05.917194 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae103bd-3124-4036-b66c-35c71e8b1e25 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:05.925687 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:05.926832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.926832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.927086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.927731 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:05.929168 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:05.943234 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:05.944780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:05.945953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:05.948986 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 448/882] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:05.951625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:05.973707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:05.974747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:05.975233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:05.975721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:05.976794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:05.977320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:05.979114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:05.990502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 435/883] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:05.991857 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.073129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.082936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e754105-bf51-4eb6-92f9-37a78cdf2fab None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.122353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.123190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.123515 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.123945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.124704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:06.125030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.126411 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c26ab1-6b7b-429d-9494-3a6f104b31da None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.128202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e5bb3-3006-4834-9ca6-2bd38426face None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.132387 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 449/884] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:06.133896 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16bf8-a2ba-4ef7-ab5c-562b8be57209 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.134397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16bf8-a2ba-4ef7-ab5c-562b8be57209 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.134878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16bf8-a2ba-4ef7-ab5c-562b8be57209 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.137120 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 450/885] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:06.138470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b16bf8-a2ba-4ef7-ab5c-562b8be57209 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.169081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.170335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.170560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.170948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.171673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:06.171985 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.173406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c77160-cfe6-44a4-910a-eaca44e2bc5d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.183228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 436/886] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:06.183228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.183228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.183228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.186749 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.201800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:06.203023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:06.276843 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b16bf8-a2ba-4ef7-ab5c-562b8be57209 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.277675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 437/887] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:06.278531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.316550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.317231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.317524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.317810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.318533 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:06.318854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.320243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea430f-2155-43ea-985b-15f9514e7c53 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.324863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 451/888] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:06.326260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc35e-520e-4644-9ce4-0722099a61b0 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.326554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc35e-520e-4644-9ce4-0722099a61b0 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.326845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21cc35e-520e-4644-9ce4-0722099a61b0 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.328313 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 452/889] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:06.328945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cc35e-520e-4644-9ce4-0722099a61b0 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.349390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.423295 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecd749-08eb-43e0-81f6-d12e025f9379 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.464861 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.465562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.465878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.466162 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.467369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] RBAC: Adding query filter params (interface=public, service_id=42438e62d07b4471b5382e3d981cf94b, region_id=RegionOne) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:06.467716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.468112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.469176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf662a-4911-4c61-a940-048f2ab4e7d9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.477399 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21cc35e-520e-4644-9ce4-0722099a61b0 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.478037 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 438/890] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/endpoints?service_id=42438e62d07b4471b5382e3d981cf94b&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:06.479381 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.479930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.480187 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.501946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:06.511999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:06.538129 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.538961 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.539611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.539611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.551601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:06.560951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:06.562257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:06.563480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 453/891] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:06.568506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.586045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 439/892] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:06.589573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.654506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.692809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b83a6-514d-482a-adf4-0b8981e69371 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.725154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e4166-23b7-4f54-ad4c-a27b83916082 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.730663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.731345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.731632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.731912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.733354 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.734568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.751287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 454/893] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:06.751287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.774230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:06.781489 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.782685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.783280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.783695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.784631 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:06.785062 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.786762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.801694 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 440/894] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:06.802832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.877930 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6f0b-9dce-42a0-bb9d-70486dc0493d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.921225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:06.922387 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:06.922845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:06.923309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:06.924201 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:06.925898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:06.941756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:06.943180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:06.944199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:06.947586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 455/895] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:06.954712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:06.962093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eddc6-9059-447c-9a4c-8ab603887c1b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:06.965030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.015602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.016470 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.017556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.017556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.029814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.031327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:07.054624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:07.055866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:07.057009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:07.057009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:07.057009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:07.057009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:07.057009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 441/896] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:07.063773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.068088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.099642 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d7556-f04e-4e3b-8ab0-5beb2c9c0208 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.143380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.144346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.144830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.145301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.146192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:07.147216 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.149047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.162187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 456/897] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:07.163830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.219466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adeb4f-7a6c-4d68-aaf9-964e3ae367b7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.268150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.268874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.269169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.269457 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.270533 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:07.270862 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.272592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.295223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 442/898] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:07.295809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.321304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf03185-112c-428a-a6d2-b543a481fe05 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.322544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.374199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.375310 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.375834 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.376335 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.377375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:07.377960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.379552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.392128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 457/899] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:07.393989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.413928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.427137 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7020d2f-7cc9-4981-a7f7-1a01fc04ea42 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.467436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.468265 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.468622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.468907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.477038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.478443 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:07.494914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:07.505743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 443/900] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:07.508021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.541076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f4ce3-ef55-4e76-9a2e-fb703fb9a991 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.589318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.590370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.590871 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.591879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.592008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.593076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] RBAC: Adding query filter params (interface=public, service_id=76ac853fa17d4949a1cc4d989c14748a, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:07.593915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.595528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.602290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 458/901] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/endpoints?service_id=76ac853fa17d4949a1cc4d989c14748a&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:07.611123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.621301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0da3c5-9429-4d3d-9446-895df17b4df5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.662331 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.663450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.663769 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.664041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.664930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:07.665456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.666986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.691722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 444/902] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:07.692799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:07.742724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.779630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf392b7-ee79-4a88-8c13-430051cb4c13 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.808057 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffff72-239a-4cd9-aa2c-44c2eed0717a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:07.820637 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.821633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.822084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.822545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.823403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.824972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:07.840860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 459/903] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/services => generated 1268 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:07.853259 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:07.853951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:07.854474 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:07.854770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:07.858217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:07.938710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:07.940496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:08.069469 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 445/904] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:07 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:26:08.081461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:08.203838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78fda-d4a0-4608-9217-59b524bb0770 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:08.249232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:08.250353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.250811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.251405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.261336 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:08.266017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:08.267708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:08.287955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:08.289372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:08.290671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:08.290671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:08.290671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:08.290671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:08.292758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 460/905] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:08.302171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:08.356333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:08.433029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989284b-479c-4093-8c3a-d2f6f93b2cd9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:08.476642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:08.477638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.478181 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.478627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.479525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:08.480007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:08.481644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:08.500738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 446/906] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:08.509549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:08.630082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25055a7-e1eb-42a9-a547-fccae051fcd6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:08.670415 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:08.671540 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.672096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.672573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.680437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:08.682097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-4c76109e-609e-4728-a7d0-90b2ac671459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:08.698656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:08.700892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:08.702900 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:08.707229 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 461/907] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:08.712990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:08.760759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869488f8-fe77-4a31-b2de-203526eb7627 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.761231 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869488f8-fe77-4a31-b2de-203526eb7627 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.761593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869488f8-fe77-4a31-b2de-203526eb7627 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.764085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 462/908] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:08.769887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.770256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.770622 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.794936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:08.796330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:08.847524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:08.855565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:08.860495 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296bba3-eef9-4e7d-826b-a691d54e2f0a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:08.888712 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 463/909] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:08.905800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:08.906775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:08.907260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:08.907792 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:08.908709 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:08.909032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:08.909561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:08.910560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:08.930560 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 447/910] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:08.940463 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:09.039442 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec42c2ac-c075-4448-b9c7-717b14897033 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:09.056562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40794854-aa7d-4e53-bacf-2edd4da67eba None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:09.087822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:09.088600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.088929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.089267 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.089995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:09.092917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbfc67-8a81-4a77-a1dc-405faa148b3a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:09.104784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:09.105480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.105772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.106054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.107597 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] RBAC: Adding query filter params (role.id=c6959c9d51794589977e42a360611102, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=b8ab51b1618f461bb8918f85b58c2793) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:09.107975 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:09.109433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b421-853f-4e7e-9b57-fa4a742d434a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:09.115845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 464/911] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:08 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:09.124844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 448/912] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:08 2022] GET /identity/v3/role_assignments?user.id=b8ab51b1618f461bb8918f85b58c2793&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=c6959c9d51794589977e42a360611102 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:09.284592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829100-63e3-49ce-8c6f-941f2df69d1a None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.284822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829100-63e3-49ce-8c6f-941f2df69d1a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.285063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4829100-63e3-49ce-8c6f-941f2df69d1a None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.287009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 465/913] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:09.293211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.293469 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.293710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.314798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:09.316085 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:09.342291 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:09.412231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 449/914] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:09.436767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829100-63e3-49ce-8c6f-941f2df69d1a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:09.487636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:09.571405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4829100-63e3-49ce-8c6f-941f2df69d1a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:09.611322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:09.612206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.612529 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.612812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.613486 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:09.614882 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:09.628305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:09.629598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:09.631605 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 466/915] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:09.635970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:09.758810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01f01-430a-4e2f-98f1-2432478ce724 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:09.796274 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:09.796984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.797583 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.797847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.798576 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:09.798887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:09.800375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:09.806531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 450/916] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:09.812864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:09.944045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ab08d-e908-470b-b654-83008565dcc9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:09.952905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:09.992960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:09.994050 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:09.994555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:09.995070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:09.996013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:09.996570 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:09.999007 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bcdc9c-f26d-47a4-b795-a856d1e98756 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:10.005033 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 467/917] 10.222.0.13 () {56 vars in 1114 bytes} [Thu May 19 20:26:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:10.017934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:10.030403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:10.134901 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b46dc9-39f6-4a85-8ffc-97f9664d4397 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:10.178384 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:10.179088 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:10.179376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:10.179656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:10.180484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:10.181932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318363a-7732-4168-b888-40f5029efe7b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:10.203932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 451/918] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:10.358756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c09cb-f4b7-407f-a952-68cf31ed7578 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:10.359138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c09cb-f4b7-407f-a952-68cf31ed7578 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:10.359331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c09cb-f4b7-407f-a952-68cf31ed7578 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:10.361092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 468/919] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:10.366967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:10.367288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:10.367540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:10.387158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:10.388364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:10.461344 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:10.490669 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 452/920] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:10.515623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c09cb-f4b7-407f-a952-68cf31ed7578 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:10.638909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c09cb-f4b7-407f-a952-68cf31ed7578 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:10.675923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:10.676790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:10.677139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:10.677434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:10.678137 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:10.679521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:10.693627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:10.695950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:10.699879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 469/921] 10.222.0.13 () {56 vars in 1132 bytes} [Thu May 19 20:26:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:10.710629 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:10.838591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffff96-0cf8-48f2-b7e9-a2dd3b68cf2d None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:10.877959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:10.878645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:10.878930 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:10.879208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:10.879935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:10.880242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:10.881671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:10.887354 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 453/922] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:10.893163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:10.932815 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:11.008707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea102a-34ec-4c34-9258-b8809b866b19 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:11.045607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:11.046341 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.046633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.046914 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.047679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:11.048000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:11.049468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c82aeb-ea69-4bae-ae34-32d1ff0eb4ef None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:11.056240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 470/923] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:10 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:11.065802 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:11.130171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:11.184614 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413abea-08ed-4f61-abd9-a126357ca2fb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:11.224833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:11.225506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.225793 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.226074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.226830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:11.228215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9143e7-3ce2-4a47-ad09-1eb83f805a17 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:11.253445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 454/924] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:11.479300 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ca28-7176-43e3-93c5-2034404c8dc3 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.479497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ca28-7176-43e3-93c5-2034404c8dc3 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.480036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ca28-7176-43e3-93c5-2034404c8dc3 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.481362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 471/925] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:11.487325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589246-22a7-45a6-bbec-630da450d5c8 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.487920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589246-22a7-45a6-bbec-630da450d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.488637 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c589246-22a7-45a6-bbec-630da450d5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.507300 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1c589246-22a7-45a6-bbec-630da450d5c8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:11.508541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1c589246-22a7-45a6-bbec-630da450d5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:11.598551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 455/926] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:11.622184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ca28-7176-43e3-93c5-2034404c8dc3 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:11.651552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bcfcc-acc9-4187-aab4-769ab2ad2dcc None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.651953 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bcfcc-acc9-4187-aab4-769ab2ad2dcc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.652144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bcfcc-acc9-4187-aab4-769ab2ad2dcc None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.654595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 456/927] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:11.659749 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.660092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.660385 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.679451 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:11.680491 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:11.713668 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:11.748172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:11.754849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ca28-7176-43e3-93c5-2034404c8dc3 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:11.762480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 457/928] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:11.785004 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:11.797982 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:11.799210 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.799706 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.800217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.813999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:11.823579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:11.824756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:11.825820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:11.828867 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 472/929] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:11.836459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:11.901332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c24c7-18ae-4cbb-ac31-be32dc346f85 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:11.942316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:11.944133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:11.944133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:11.944133 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:11.945017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:11.946620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:11.962021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:11.963501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:11.964791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:11.967629 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 458/930] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:11.973281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:11.985720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bcce2-ad07-4cdc-bf91-2353ac6a88a2 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.008423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.035840 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.037330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.037848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.038566 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.039534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.040243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.042682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.061232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 473/931] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:12.063016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.089419 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbafb3-111f-469c-9826-d54ecb0f7974 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.112908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.137848 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264702c-a887-489f-856d-ad7069698363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.139936 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264702c-a887-489f-856d-ad7069698363 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.140685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264702c-a887-489f-856d-ad7069698363 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.141157 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264702c-a887-489f-856d-ad7069698363 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.142759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264702c-a887-489f-856d-ad7069698363 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.143243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264702c-a887-489f-856d-ad7069698363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.145894 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0264702c-a887-489f-856d-ad7069698363 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.153855 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 459/932] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:12.159975 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264702c-a887-489f-856d-ad7069698363 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.207932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d5fb64-83a6-4604-be23-ffc604ca99c1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.250749 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.252547 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.253013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.253394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.266568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.268008 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.285570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:12.287308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:12.288794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:12.290056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:12.290056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:12.290056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:12.290056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:12.296227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 474/933] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:12.297663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.301075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264702c-a887-489f-856d-ad7069698363 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.353363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.354466 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.354999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.356116 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.358365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.359137 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.360938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.369124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 460/934] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:12.375760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.395985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.440625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc086c-745b-4fb5-928c-99668281b0e1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.483250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.483954 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.484246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.484588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.485506 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.485702 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.487103 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.497299 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41fa62-75bd-4188-b26e-0057e1b806de None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.506089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 475/935] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:12.507217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.542892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.543961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.545041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.545558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.546753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] RBAC: Adding query filter params (interface=public, service_id=6b28b0fe166845d78153464bd692cf31, region_id=RegionOne) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.547286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.548984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.559767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 461/936] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/endpoints?service_id=6b28b0fe166845d78153464bd692cf31&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:12.562212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.590507 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.625652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d508d18-779a-4ce3-bebf-3e7dcbacff2a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.662727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:12.670707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.671616 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.671902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.672188 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.681045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.682610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.684835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ce554-2e39-4bd5-8ddc-2fe392ba1c7b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:12.700098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:12.702069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:12.703200 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:12.708139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 476/937] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:12.709232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.729112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.730581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.731150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.731642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.732742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.734400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e406bfe-4de2-425c-a8d1-38a3bfa94da1 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.748566 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 462/938] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:12.815559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f8f09-5852-4195-8419-a0a693f9c139 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.816205 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f8f09-5852-4195-8419-a0a693f9c139 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.816757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f8f09-5852-4195-8419-a0a693f9c139 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.818799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 463/939] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:12.827325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.827325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.827325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.835135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8aefe-5022-4fde-836f-00d7b7387011 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:12.848206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:12.849656 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:12.892116 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:12.893260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:12.893777 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:12.894287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:12.895396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:12.895970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:12.897692 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:12.919026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 477/940] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:12.925346 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.964934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 464/941] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:12.992537 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:12.993593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.022226 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.055785 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0273431-c9a4-40dd-a430-29497ff134d5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.104974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.105982 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.106278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.106564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.118133 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.119720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:13.141112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:13.142276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:13.143370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:13.143370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:13.143370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:13.143370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:13.143370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 478/942] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:13.145517 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8b610-7638-46d4-a67f-cd4a75243eec None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.150012 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.195764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.196492 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.196847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.197136 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.197991 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.199536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.201840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: WARNING keystone.server.flask.common [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:26:13.232429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.sql.core [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:26:13.232429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:26:13.232429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [parameters: {'id': 'ec716696276d41f5823fe4afb636842f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:26:13.232429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277574) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec716696276d41f5823fe4afb636842f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec716696276d41f5823fe4afb636842f. May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application conn.query(q) May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:26:13.244023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application result.read() May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:26:13.245097 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application next(self.gen) May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application yield resource May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application next(self.gen) May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise self.value May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application session.commit() May 19 20:26:13.246375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self.session.flush() May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application compat.raise_( May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:26:13.247633 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:26:13.250048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application conn.query(q) May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application result.read() May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:26:13.252083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [parameters: {'id': 'ec716696276d41f5823fe4afb636842f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.262217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:13.264883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ec716696276d41f5823fe4afb636842f. May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 465/943] 10.222.0.13 () {60 vars in 1140 bytes} [Thu May 19 20:26:12 2022] POST /identity/v3/projects => generated 262 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:13.266505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.273835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a896116-05b2-4e7a-8e1c-a15a1e6415f1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.277423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.328421 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.329512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.329818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.330107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.330862 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:13.331184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.332713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.350694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 479/944] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:13.357735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.373858 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.402420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a56f-cbc1-4800-b728-bb7729e247e3 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.441685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.442532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.442824 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.443112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.452381 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.454179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:13.471575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:13.472822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:13.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:13.478744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 466/945] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:13.479840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.493495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09d782-2db2-4142-92e8-0dab609802db None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.533818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.534946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.535475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.535916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.544005 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.546683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:13.565636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:13.566816 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:13.567931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:13.571024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 480/946] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:13.579192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.614559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.618973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259355df-a0f4-42ec-9783-03372baab926 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.651272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.659298 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.660011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.660306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.660626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.661416 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:13.661733 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.663216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9ff3-cd61-496e-84e0-3e109461e961 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.694915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 467/947] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:13.694915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb647-aac9-436d-b923-d4b0e93c9edc None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.694915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb647-aac9-436d-b923-d4b0e93c9edc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.694915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfb647-aac9-436d-b923-d4b0e93c9edc None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.694915 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 468/948] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:13.697447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.697822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.698186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.720282 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d1fb9-9f46-4870-a70c-d3bfb583bc0f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:13.724207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:13.725951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:13.766305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650451-5207-4132-969e-b0c60287715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:13.767575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650451-5207-4132-969e-b0c60287715b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:13.769327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650451-5207-4132-969e-b0c60287715b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:13.770063 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650451-5207-4132-969e-b0c60287715b None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:13.771202 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650451-5207-4132-969e-b0c60287715b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:13.771628 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650451-5207-4132-969e-b0c60287715b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:13.773476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650451-5207-4132-969e-b0c60287715b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:13.792630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 481/949] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:13.798227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650451-5207-4132-969e-b0c60287715b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.832509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 469/950] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:13.870482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:13.931806 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:13.960497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650451-5207-4132-969e-b0c60287715b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:14.009597 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:14.010507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:14.010792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:14.011073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/b8ab51b1618f461bb8918f85b58c2793/roles/c6959c9d51794589977e42a360611102` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:14.013512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=b8ab51b1618f461bb8918f85b58c2793, role_id=c6959c9d51794589977e42a360611102)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:14.015065 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:14.031639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d1788-0820-4d90-a142-3d9b686cc3d5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:14.034296 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 482/951] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:26:13 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/b8ab51b1618f461bb8918f85b58c2793/roles/c6959c9d51794589977e42a360611102 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:26:14.063636 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:14.079438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:14.080289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:14.080612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:14.080901 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:14.081589 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:14.083046 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:14.096988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:14.098282 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:14.099338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 470/952] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:14.105396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:14.237477 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeb40f-ca85-418d-8290-4a41e46822a5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:14.280304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:14.281478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:14.281936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:14.282395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:14.283304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] RBAC: Adding query filter params (name=compute) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:14.283787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:14.285523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991526d-edd5-4b8c-bc2d-def5bf2242eb None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:14.292078 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 483/953] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:14.298777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:14.460428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9b332-082f-4253-a669-639ebcc27915 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:14.488017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:14.488675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:14.488962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:14.489243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:14.489979 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] RBAC: Adding query filter params (type=compute) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:14.490293 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:14.491715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f75f5-4b86-4039-a9a3-733c4605526a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:14.498335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 471/954] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:15.326243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a896ea-b306-4534-9b11-37d6538f4a6a None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:15.326808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a896ea-b306-4534-9b11-37d6538f4a6a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:15.327360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a896ea-b306-4534-9b11-37d6538f4a6a None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:15.329263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 484/955] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:15.335175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:15.335686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:15.335800 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:15.354829 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:15.356047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:15.407762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:15.433230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 472/956] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:15.456875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a896ea-b306-4534-9b11-37d6538f4a6a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:15.466781 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:15.573143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a896ea-b306-4534-9b11-37d6538f4a6a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:15.610748 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:15.611481 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:15.611768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:15.612050 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:15.612749 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:15.614124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:15.627538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:15.628820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:15.629827 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 485/957] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:15.635747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:15.750318 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7ccef-ec72-4c4d-b2ed-9b9526b59dea None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:15.787702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:15.788455 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:15.788746 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:15.789024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:15.789754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:15.790065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:15.791467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:15.798044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 473/958] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:15.803612 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:15.891478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:15.926026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467b4b1-8c25-44bd-8733-e7078dc084e2 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:15.964647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:15.965325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:15.965611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:15.965893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:15.966682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:15.966995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:15.968373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5fbd3-e7e3-4816-8f92-3e4c3b983983 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:15.974055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 486/959] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:15.979545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.036864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:16.100692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c7f9-8cbc-4e22-8eba-d27b290fadc2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:16.138478 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:16.139198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.139484 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.139764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.140558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:16.142004 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d2919-343a-43ae-bc2d-466de67eed22 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:16.164215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 474/960] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:16.243678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9648b1-85d5-4fba-9d9a-2e0a7efaf434 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.244371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9648b1-85d5-4fba-9d9a-2e0a7efaf434 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.244946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9648b1-85d5-4fba-9d9a-2e0a7efaf434 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.247057 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 487/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:16.253948 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07b3f5-265a-4551-84e1-92edf159b172 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.254284 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07b3f5-265a-4551-84e1-92edf159b172 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.254520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07b3f5-265a-4551-84e1-92edf159b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.275889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-2c07b3f5-265a-4551-84e1-92edf159b172 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:16.277575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-2c07b3f5-265a-4551-84e1-92edf159b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:16.376739 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 475/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:16.399633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9648b1-85d5-4fba-9d9a-2e0a7efaf434 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.523459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9648b1-85d5-4fba-9d9a-2e0a7efaf434 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:16.559184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:16.560372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.560695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.560974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.561689 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:16.563187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:16.564798 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: WARNING keystone.server.flask.common [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:26:16.572592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:16.580910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.sql.core [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:26:16.580910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:26:16.580910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [parameters: {'user_id': 'cacdc6df1cfa443aa74bb48cb422846f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:26:16.580910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=277575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:26:16.589412 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45209-a7e9-4ca2-bdac-a32ab72565cb None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.589827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45209-a7e9-4ca2-bdac-a32ab72565cb None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.589957 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b45209-a7e9-4ca2-bdac-a32ab72565cb None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application conn.query(q) May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result.read() May 19 20:26:16.590045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:26:16.591645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application next(self.gen) May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application yield resource May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application next(self.gen) May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise self.value May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application session.commit() May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:26:16.596489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.session.flush() May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application compat.raise_( May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:26:16.598482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:26:16.601009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application conn.query(q) May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application result.read() May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:26:16.603155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cacdc6df1cfa443aa74bb48cb422846f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:16.605479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:26:16.608052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:26:16.610306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.611067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 476/964] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:16.611214 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 488/964] 10.222.0.13 () {60 vars in 1131 bytes} [Thu May 19 20:26:16 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:16.611214 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.611479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.611479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.611479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.621755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:16.623840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:16.652488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:16.724577 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085b68d-8016-4a47-b4c5-afdbef75129c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:16.728750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 477/965] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:16.745100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.764653 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:16.765475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.765763 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.766051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.774459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:16.776189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:16.793095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:16.794955 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:26:16.796744 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:16.800456 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 489/966] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:26:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:16.800995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.879453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359232f-c306-4002-85da-246cf8bad5f9 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:16.880563 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:16.925447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:16.926698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.927151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.927609 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.932424 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4edb09-914b-4dd5-acee-0978f2656c0f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:16.945459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:16.957383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:16.958840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:16.960069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:16.964127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 478/967] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:16.965083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:16.982449 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:16.983944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:16.984482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:16.985166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:16.986791 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:16.987575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:16.989386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:16.989695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b76dc4-1a7a-4b7f-b5ed-4de0209a5557 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.045820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 490/968] 10.222.0.13 () {56 vars in 1101 bytes} [Thu May 19 20:26:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:17.048421 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a353-cfb7-4cda-85a6-38ebfa9d06ee None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.049197 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a353-cfb7-4cda-85a6-38ebfa9d06ee None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.049681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9a353-cfb7-4cda-85a6-38ebfa9d06ee None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.057024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 491/969] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:17.066895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.067264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.067926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.093226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:17.094001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335048-923e-4035-948c-3ea6f6f8e52e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:17.094891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:17.140107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:17.141168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.141558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.141942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.142842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:17.143153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:17.144625 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.160350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 479/970] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:17.161359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:17.180590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 492/971] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:17.210535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:17.269205 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:17.280109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:17.310574 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2b4b4-bfeb-44b7-8e02-54b7ee67211b None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:17.347550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dce07a-4ba6-4a6b-b7a8-8a246b82941b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:17.356438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:17.358513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.359030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.359539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.372077 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:17.373886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:17.396360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:17.397928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:17.399112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:17.399112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:17.399112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:17.399112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:17.399393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:17.399393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.399393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.399393 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.399718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:17.400864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.402983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 480/972] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:17.405540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:17.418452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 493/973] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:17 2022] POST /identity/v3/services => generated 242 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:26:17.553501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325d772-a66f-4d95-b13a-8eda66146529 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:17.588642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:17.610970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:17.611659 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.611947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.612225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.613038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:17.613967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:17.614795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.630153 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 481/974] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:17.635704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:17.752510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82a41f-1bfb-4f88-9491-660436f533f9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:17.756789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:17.793905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:17.795242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:17.795524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:17.795802 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:17.803273 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:17.812877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:17.831558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:17.832987 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:26:17.834148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:17.835558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 494/975] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:26:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:17.839532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:17.963110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860aa72-a9d9-4bd0-bce3-b6e073602e55 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.003940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.004677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.004969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.005250 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.006160 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:18.006481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.007935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.039912 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 482/976] 10.222.0.13 () {56 vars in 1149 bytes} [Thu May 19 20:26:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:18.041374 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.152339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c2450-12da-4b5f-a306-0b208effa342 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.190384 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.191055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.191343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.191620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.227274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:18.270872 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.272321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.383820 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 495/977] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:18.385551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.442107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:18.503562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30842f8-c404-497b-85f1-3d0dacd01605 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.540134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.540980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.541273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.541558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.554478 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.555978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5b5b1fd5-6513-4596-8141-ebf85bda54df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:18.576123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:18.577309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:18.578415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:18.578415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:18.578415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:18.578415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:18.579271 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 483/978] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:18.583048 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.604623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30aa0-69eb-421b-a680-3ce02c4676ac None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.605059 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30aa0-69eb-421b-a680-3ce02c4676ac None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.605431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e30aa0-69eb-421b-a680-3ce02c4676ac None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.607508 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 484/979] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:18.613148 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.613446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.613738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.631403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:18.632639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:18.658118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:18.704788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8249d-70bd-40c1-8c4d-e8e46b6a69f6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.724502 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 485/980] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:18.743842 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.744557 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.744853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.745143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.745891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:18.746207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.746570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.747638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.765971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 496/981] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:18.768080 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.776987 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:18.873897 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a7d700-db53-4670-8e7e-5de17979771a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.900093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba312c3-9814-41bf-9fc7-039b2969b090 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:18.912174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.913080 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.913369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.913632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.914352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.915773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:18.929376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:18.930951 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:18.932156 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:18.935841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 486/982] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:18.937288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:18.937518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:18.951259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:18.952306 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:18.952828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:18.953295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:18.961028 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:18.962427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:18.981347 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:18.983167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:18.984253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 497/983] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:18.989440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:19.046816 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:19.058064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4178d7-09e4-4cb1-b675-293c76482d4e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.099085 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.099762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.100052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.100333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.101074 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:19.101386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.102768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.103723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4318b-b0c2-4ce7-b4b6-09a36b3f7ec7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.109332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 487/984] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:19.115026 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:19.144979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.145682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.145974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.146265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.147064 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:19.147386 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.148858 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.162398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 498/985] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:19.168562 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:19.220376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:19.235323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f93b86-cd8e-43c3-a050-bc13886c45bb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.272991 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.274089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.274546 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.274980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.275886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:19.276509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.278285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.284590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a79ee4-2b42-4ebd-af77-482db540c122 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.285682 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 488/986] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:19.291508 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:19.315226 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:19.324704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.326153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.326448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.326728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.327609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] RBAC: Adding query filter params (role.id=c6959c9d51794589977e42a360611102, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=b8ab51b1618f461bb8918f85b58c2793) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:19.327936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.329471 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.336085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 499/987] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:19 2022] GET /identity/v3/role_assignments?user.id=b8ab51b1618f461bb8918f85b58c2793&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=c6959c9d51794589977e42a360611102 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:19.425588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1be95-7fb7-4c2f-82d9-df24fc769f6c None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.468554 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.469355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.469541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.469806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.470676 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] RBAC: Adding query filter params (interface=admin, service_id=6b28b0fe166845d78153464bd692cf31, region_id=RegionOne) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:19.470941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.472369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc43715-3203-4bca-b3b7-dfe14b1d4349 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.483511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 489/988] 10.222.0.13 () {56 vars in 1249 bytes} [Thu May 19 20:26:19 2022] GET /identity/v3/endpoints?service_id=6b28b0fe166845d78153464bd692cf31&interface=admin®ion_id=RegionOne => generated 193 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:19.485641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:19.614794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d4f74-596f-4cf6-9ad7-b5b4ecd1fd9c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:19.625276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:19.653413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:19.654958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.655448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.655953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.656911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:19.658480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bc348-c024-4f9c-b23b-14b4e36d9bbc None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:19.665968 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 500/989] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:19 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:19.755807 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c814-c908-4943-bd31-52beeee011e3 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.756025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c814-c908-4943-bd31-52beeee011e3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.756283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64c814-c908-4943-bd31-52beeee011e3 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.757833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 490/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:19.765128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:19.765453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:19.765883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:19.785056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:19.786299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:19.872199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 501/991] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:19.896357 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c814-c908-4943-bd31-52beeee011e3 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:20.002036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:20.034023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64c814-c908-4943-bd31-52beeee011e3 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:20.071984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:20.072892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:20.073182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:20.073461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] PATH_INFO: `/v3/services/compute` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:20.074127 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:20.076198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:20.088914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:20.090113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:20.091821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 491/992] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:20.109568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:20.230805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f780d0-8e2f-42e1-9cb6-31fdf4301806 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:20.232465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:20.270833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:20.271868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:20.272335 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:20.272822 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:20.273738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:20.274246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:20.275833 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:20.282046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 502/993] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:20.287920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:20.410577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574e55d-4075-4035-acb4-0edfb853d560 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:20.448230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:20.448974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:20.449267 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:20.449606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:20.450334 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:20.450715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:20.452109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:20.462396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 492/994] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:20.463718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:20.598962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d468cb-d1a3-413b-b5d4-c6955bc3bd05 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:20.637516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:20.640495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:20.640495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:20.640495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:20.640790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] RBAC: Adding query filter params (interface=public, service_id=5e04f3c8d4a84c91a157731991477114, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:20.640985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:20.642218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4b7fc-4b6f-4ef7-bd25-a37223e55ca4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:20.648538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 503/995] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:26:20 2022] GET /identity/v3/endpoints?service_id=5e04f3c8d4a84c91a157731991477114&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:20.654670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:20.669896 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:20.771089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203bf86-c862-40a2-aa10-83461635d4e2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:20.809545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:20.811124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:20.811262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:20.811489 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:20.812109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:20.813527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c817bbc-54b3-4820-afd3-a6bc15bfcb05 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:20.828448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 493/996] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:20 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:21.697278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1a5e-1d9f-4812-8103-1824bc10116e None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:21.697878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1a5e-1d9f-4812-8103-1824bc10116e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:21.697878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b1a5e-1d9f-4812-8103-1824bc10116e None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:21.699391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 504/997] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:21.705113 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36148-47f4-4927-b490-c8ed4c97babd None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:21.705628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36148-47f4-4927-b490-c8ed4c97babd None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:21.705723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e36148-47f4-4927-b490-c8ed4c97babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:21.733998 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c5e36148-47f4-4927-b490-c8ed4c97babd None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:21.735230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c5e36148-47f4-4927-b490-c8ed4c97babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:21.808459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 494/998] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:21.835123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b1a5e-1d9f-4812-8103-1824bc10116e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:21.882793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:21.960574 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b1a5e-1d9f-4812-8103-1824bc10116e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:21.999758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.000606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.000894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.001177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] PATH_INFO: `/v3/services/image` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.001853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.003269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-bac13dcf-7074-4c37-a5dd-70d6e3a9cbc9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:22.016143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:22.017703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:22.019410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.020878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60b60f-27b6-465c-b6f1-a34239d44a22 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.020878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60b60f-27b6-465c-b6f1-a34239d44a22 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.020878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60b60f-27b6-465c-b6f1-a34239d44a22 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.020878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 495/999] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:26:22.023788 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 505/1000] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:26:21 2022] GET /identity/v3/services/image => generated 86 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:22.023988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60b60f-27b6-465c-b6f1-a34239d44a22 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.025174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.025473 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.025760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.027398 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:22.043603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:22.045670 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:22.122157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 506/1001] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:22.147701 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.147939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60b60f-27b6-465c-b6f1-a34239d44a22 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.179333 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:22.199564 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.200259 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.201171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.201602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.202559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] RBAC: Adding query filter params (name=image) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:22.202877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.204281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.210043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 496/1002] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:22.215723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.271756 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80536ba2-f88c-4a90-aa18-d97586d4cc2f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.290155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:22.314381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.315787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.316257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.316868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.317830 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.319211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:22.333225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:22.334488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:22.335520 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 507/1003] 10.222.0.13 () {56 vars in 1126 bytes} [Thu May 19 20:26:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:22.340691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.340921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c6328-a17a-4852-b1d9-6ec6d049346a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.381823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.382503 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.382788 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.383066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.383775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] RBAC: Adding query filter params (type=image) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:22.384086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.385506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.391642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 497/1004] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:22.458979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:22.475632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ee20a-0f61-4d1d-ad78-5862660c041c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.516916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.517604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.517899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.518182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.518912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:22.519227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.520652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.531223 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 508/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:22.532643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.656682 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b700f7-facc-4d68-be7f-fd2471ed83b0 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.689968 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:22.696292 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.696999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.697288 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.697567 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.698286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:22.698596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.700060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0beb428-c561-4d6a-9d8e-5dffe872d067 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.708547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 498/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Thu May 19 20:26:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:22.712250 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:22.823610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb3a3a-099b-42b0-8f7d-9e6a45faee2f None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:22.868439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:22.868937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:22.869228 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:22.869676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:22.870305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:22.871737 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a6180-d8dd-4be8-a477-5dc1989dfb85 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:22.896439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 509/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:23.232831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785ec8c-b55f-44fb-84d7-cf2d3f89392b None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:23.233179 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785ec8c-b55f-44fb-84d7-cf2d3f89392b None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:23.233527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785ec8c-b55f-44fb-84d7-cf2d3f89392b None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:23.235233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 499/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:23.240853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:23.241152 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:23.241437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:23.259380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:23.260624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:23.269065 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:23.345767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 510/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:23 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:23.369210 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785ec8c-b55f-44fb-84d7-cf2d3f89392b None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:23.487974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785ec8c-b55f-44fb-84d7-cf2d3f89392b None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:23.527500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:23.528312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:23.529247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:23.529532 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:23.530206 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:23.531611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:23.543973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:23.546350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:26:23.547437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:23.549208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 500/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:23.552488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:23.668226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34f781-70b8-4239-9088-93334fe80770 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:23.707561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:23.708251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:23.708576 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:23.708862 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:23.709596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:23.709913 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:23.711311 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:23.717093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 511/1011] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:23.722659 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:23.787704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:23.833587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e247defd-5406-4dda-8c71-0c81fd07cf1f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:23.877056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:23.877736 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:23.878030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:23.878316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:23.879040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] RBAC: Adding query filter params (type=compute) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:23.879358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:23.880782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c97e51-ff69-4820-8a93-53b6642c6e74 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:23.886050 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 501/1012] 10.222.0.13 () {56 vars in 1112 bytes} [Thu May 19 20:26:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:23.891838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:23.894750 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:24.001409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6413e-a201-466a-a75b-ed819c6cb9f5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:24.038707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:24.039299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:24.039590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:24.039877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:24.040674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:24.042065 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b119e-bfb3-478c-b3a4-50e417a93558 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:24.062190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 512/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:24.737754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a24d4-40cb-4c62-a238-bc41ceb65ecc None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:24.738327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a24d4-40cb-4c62-a238-bc41ceb65ecc None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:24.738327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a24d4-40cb-4c62-a238-bc41ceb65ecc None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:24.739941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 502/1014] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:24.745614 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf146646-d37d-47a3-bd25-4280c8842280 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:24.746183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf146646-d37d-47a3-bd25-4280c8842280 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:24.746183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf146646-d37d-47a3-bd25-4280c8842280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:24.764829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-cf146646-d37d-47a3-bd25-4280c8842280 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:24.765928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-cf146646-d37d-47a3-bd25-4280c8842280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:24.824264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:24.852935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 513/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:24 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:24.876119 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a24d4-40cb-4c62-a238-bc41ceb65ecc None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:24.992056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a24d4-40cb-4c62-a238-bc41ceb65ecc None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:25.020287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:25.034773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:25.035479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:25.035771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:25.036057 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:25.036862 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:25.038306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7de36-82ab-439a-9c84-9d2e71bc2e99 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:25.072426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 503/1016] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:24 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:25.216879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd061ac-b352-4a1b-9649-1cb5b4296bb1 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:25.217290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd061ac-b352-4a1b-9649-1cb5b4296bb1 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:25.217641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd061ac-b352-4a1b-9649-1cb5b4296bb1 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:25.219534 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 514/1017] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:25.225347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efbad-9983-4660-9393-96dfd7341aed None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:25.225746 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efbad-9983-4660-9393-96dfd7341aed None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:25.225969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efbad-9983-4660-9393-96dfd7341aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:25.244954 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-681efbad-9983-4660-9393-96dfd7341aed None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:25.246086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-681efbad-9983-4660-9393-96dfd7341aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:25.337742 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 504/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:25.361265 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd061ac-b352-4a1b-9649-1cb5b4296bb1 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:25.489985 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd061ac-b352-4a1b-9649-1cb5b4296bb1 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:25.525285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:25.526082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:25.526404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:25.526742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:25.528149 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:25.530192 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5a928-a5ff-4d73-9136-2aa6f1cc1a46 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:25.537052 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 515/1019] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:25.538173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efbad-9983-4660-9393-96dfd7341aed None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:25.649242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efbad-9983-4660-9393-96dfd7341aed None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:25.662101 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:25.698902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:25.699581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:25.699872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:25.700154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:25.701996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:25.703501 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b48d31-16d4-462c-beb4-ed725089d91a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:25.734517 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 505/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:25 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:26.519071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38416-9c8b-4637-8a62-36089d2ce87d None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:26.519673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38416-9c8b-4637-8a62-36089d2ce87d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:26.519673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb38416-9c8b-4637-8a62-36089d2ce87d None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:26.521215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 516/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:26.526624 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:26.527139 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:26.527236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:26.544768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:26.545986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:26.627996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 506/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:26.651618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb38416-9c8b-4637-8a62-36089d2ce87d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:26.734685 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:26.801641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb38416-9c8b-4637-8a62-36089d2ce87d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:26.850479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:26.851667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:26.852035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:26.852513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:26.866427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:26.877884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:26.879235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:26.886269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 517/1023] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:26.889089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.010408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:27.023423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51b6c-83c0-4d02-b205-9b1b2b2ffcbb None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.063559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.064175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.064467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.064999 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.065765 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:27.066075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.067467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.080924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 507/1024] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:27.086853 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.260951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d977-53ff-40c3-8878-f49f1e35a572 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.280898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:27.309248 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.311413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.311413 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.311638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.329752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.331141 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-185308f6-cb9a-4c06-85a5-77fd1d779d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:27.355800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:27.358030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:27.359124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:27.359124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:27.359124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:27.359124 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:27.360476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 518/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:27.364989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.454654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860172-a1ac-47f8-9b11-cdb813bf4ecf None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.455114 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860172-a1ac-47f8-9b11-cdb813bf4ecf None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.455590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860172-a1ac-47f8-9b11-cdb813bf4ecf None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.458104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 519/1026] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:26:27.463647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e35db-0d00-424c-af51-672840550768 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.463939 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e35db-0d00-424c-af51-672840550768 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.464217 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e35db-0d00-424c-af51-672840550768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.485101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-6f8e35db-0d00-424c-af51-672840550768 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:27.486580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-6f8e35db-0d00-424c-af51-672840550768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:27.512164 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b788fa-8fec-49e2-80a5-28597a25182e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.555537 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.556623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.557119 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.557600 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.558893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:27.559423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.562550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.587042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 508/1027] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:27.588345 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.589710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 520/1028] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:27.607697 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:27.614633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e35db-0d00-424c-af51-672840550768 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.706822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a9a0e-cb3f-44cf-98fd-8846be367031 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.730121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:27.740445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e35db-0d00-424c-af51-672840550768 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.746936 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.748372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.748640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.748929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.757599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.759144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:27.780920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:27.782242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:27.784269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 509/1029] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:27.786016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.786967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.787453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.787784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] PATH_INFO: `/v3/services/image` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.788390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.788572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.790015 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:27.806568 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:27.808581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:27.810447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 521/1030] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:27.816037 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:27.886428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:27.927628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713f446-ab01-4ada-a2b5-9cd8bcb28d89 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.948887 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391f795-c4f9-43d7-b087-8dd94dfb34ad None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:27.971621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.972337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.972777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.973056 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.974064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:27.974482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.976107 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.993652 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:27.994339 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:27.994634 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:27.994920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:27.995645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] RBAC: Adding query filter params (name=image) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:27.995967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:27.997966 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:27.998883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 510/1031] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:28.004937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.005270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 522/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:28.010891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.037745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.127790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb0398-0f0d-4c41-890c-33796ffacdd0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.130941 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef89f-4b67-4673-8e78-eb4f4648c123 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.164770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.166089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.166832 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.167141 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.167435 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.168159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] RBAC: Adding query filter params (type=image) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:28.168540 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.169969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a943a6a-8252-4a50-8cee-576e4a2390d9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.171510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.172185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.172496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.172783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.175099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 523/1033] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:28.177491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3481-b681-48d2-b983-7c2fdbe11c09 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.177491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3481-b681-48d2-b983-7c2fdbe11c09 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.177491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe3481-b681-48d2-b983-7c2fdbe11c09 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.184452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 524/1034] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:28.184452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3481-b681-48d2-b983-7c2fdbe11c09 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.259702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.261229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.282528 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.300215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe3481-b681-48d2-b983-7c2fdbe11c09 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.336620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.337412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.337711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.337997 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.338969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] RBAC: Adding query filter params (interface=public, service_id=4c4237c820dc49728b85d9ecc54758c5, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:28.339158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.340600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ab1ac-90f3-44da-a2c0-04cb94e3c5b3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.347261 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 525/1035] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/endpoints?service_id=4c4237c820dc49728b85d9ecc54758c5&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:28.348579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.348873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.349156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.365826 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:28.367037 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:28.371323 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 511/1036] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:26:28.373064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.408878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.447543 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 526/1037] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:28.448578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.485700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e0c82-f7fb-4ba0-b72a-cc25e0752809 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.528488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.529408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.529888 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.530317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.531066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.532756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.536239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.544387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 512/1038] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:26:28.546552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.570101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a5fd2-b134-4b0b-bd1b-6142acdb8fac None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.615779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4748-8340-465b-a846-8264e616693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.616761 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4748-8340-465b-a846-8264e616693b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.617054 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4748-8340-465b-a846-8264e616693b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.617340 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4748-8340-465b-a846-8264e616693b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.631198 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a4748-8340-465b-a846-8264e616693b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.633020 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a4748-8340-465b-a846-8264e616693b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-498a4748-8340-465b-a846-8264e616693b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:28.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:28.655488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:28.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:28.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:28.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:28.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:28.656630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 527/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:28.663158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4748-8340-465b-a846-8264e616693b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.686946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.694227 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021c822-22a2-4bf1-9556-a2cb8f9b7599 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.738895 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.739728 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.740024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.740314 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.741614 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.743151 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.744903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 513/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:28.752940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.813256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4748-8340-465b-a846-8264e616693b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.847801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:28.854624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.855321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.855683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.855980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.857654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:28.858117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.859769 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.876593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 528/1041] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:28.881183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:28.887643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e08963-e927-43ac-a819-7408e0ecb1ae None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:28.937799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:28.938465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:28.938892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:28.939034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:28.940033 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:28.941573 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:28.967199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 514/1042] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:26:28 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:29.000752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ecd96-85dc-477e-ab99-8cf323ca0ef4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:29.042225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:29.043051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:29.043343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:29.043627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:29.052451 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:29.054100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:29.071847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:29.073072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:29.074218 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:29.076849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 529/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:29.079580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:29.105608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:29.208705 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f4f39-2bb0-4904-b100-f51e0253fd98 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:29.252775 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:29.253464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:29.253754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:29.254038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:29.254846 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:29.255167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:29.256680 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adab10-3f13-451e-b27d-3481f9a96b5a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:29.272196 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 515/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:29.277975 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:29.340790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:29.395035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b0dff-7872-411e-84c9-20c5405078d5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:29.431793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:29.432495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:29.432786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:29.433066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:29.433927 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] RBAC: Adding query filter params (role.id=c6959c9d51794589977e42a360611102, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:29.434245 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:29.436422 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466d7f2-e35c-4656-8805-ec55864a736e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:29.443118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 530/1045] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:29 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=c6959c9d51794589977e42a360611102 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:30.900287 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea08-f1ef-4245-a86d-51b90ed044da None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:30.900868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea08-f1ef-4245-a86d-51b90ed044da None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:30.900868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea08-f1ef-4245-a86d-51b90ed044da None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:30.902336 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 516/1046] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:30.907935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:30.908390 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:30.908522 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:30.925988 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:30.927195 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:31.007130 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 531/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:31.031556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcea08-f1ef-4245-a86d-51b90ed044da None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.127940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.152415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcea08-f1ef-4245-a86d-51b90ed044da None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.193789 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.195017 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.195431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.195838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] PATH_INFO: `/v3/services/image` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.196876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:31.199356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d1e8a069-caf7-40e8-ae2c-a80523597ecb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:31.213686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:31.215422 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:26:31.217365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:31.220750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 517/1048] 10.222.0.13 () {56 vars in 1105 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:31.221941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.276071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.339659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262549de-1753-411f-92b2-72b2d987ed76 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.361368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8029b0-c4fa-4674-9d0a-4b245055e210 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.361679 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8029b0-c4fa-4674-9d0a-4b245055e210 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.362176 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8029b0-c4fa-4674-9d0a-4b245055e210 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.363612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 518/1049] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:31.368932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.369512 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.370014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.381264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.381870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.382163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.382452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.383177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] RBAC: Adding query filter params (name=image) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:31.383496 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:31.385395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:31.390786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 532/1050] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:31.391285 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:31.392687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:31.396589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.478044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 519/1051] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:31.502327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.520104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcb8c0-4bb6-409e-a34f-fecabbc74081 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.532512 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.562552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.563263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.563553 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.563836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.564865 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] RBAC: Adding query filter params (type=image) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:31.565188 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:31.566650 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:31.572392 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 533/1052] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:31.579142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.607424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.636453 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6453db-4d48-46df-aa5f-612177bcaf49 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.697297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.698115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.698406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.698698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.710888 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:31.712773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e4b79-4049-4f09-b3c0-354c18877d34 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:31.723730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:31.725018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:31.726048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 520/1053] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:31.731007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.756325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.757351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.757644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.757929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.758683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:31.760133 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a3929-ee4d-42da-b3b8-cd764028a44c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:31.781937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 534/1054] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:26:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:26:31.783294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade71c2c-9eb0-4b36-9402-bc9f7282731c None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.783596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade71c2c-9eb0-4b36-9402-bc9f7282731c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.783894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade71c2c-9eb0-4b36-9402-bc9f7282731c None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.785405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 535/1055] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:31.791324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.791655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.791899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.812817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:31.814084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:31.844629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.867039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e299f-7873-4b35-8d7c-b874119e16a7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:31.908324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:31.924451 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 536/1056] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:31.928062 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:31.929065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:31.929217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:31.929649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:31.930542 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:31.930932 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:31.932544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:31.939269 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:31.943405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 521/1057] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:31.951091 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.064505 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef89d7a-5676-43ae-b7a9-8997dc3ba58e None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.074707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b05f3-05c3-4b12-b007-9d95d9579cb2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.103852 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.104723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.105014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.105487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.113996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.114884 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.115177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.115462 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] PATH_INFO: `/v3/users/7078527adc134853a8cf35336771532a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.117491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:32.126539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:32.127851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:32.128978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 537/1058] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:32.134711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.136878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=7078527adc134853a8cf35336771532a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.138562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.140856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 522/1059] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:31 2022] GET /identity/v3/users/7078527adc134853a8cf35336771532a => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:32.146141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.156062 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.199855 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.255559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb2e71-eafb-4933-a4dc-e6a6dcdd4ab9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.257086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328f12-ab02-4183-aa58-f0451107d610 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.296465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.297129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.297416 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.297757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.298403 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.299086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.299373 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.299654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.300523 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:32.300844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.303308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.318620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 538/1060] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:32.319740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.381073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.382540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.409561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.433995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570c75c-c63a-4b7c-9bdd-058713f90967 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.443249 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.474146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.474864 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.475150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.475432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.486615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.488066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:32.507177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:32.508585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:32.509893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:32.509893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:32.509893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:32.509893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:32.513878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 539/1061] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:32.514794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.516260 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 523/1062] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:26:32.521828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.645249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd6fc-5e4f-432c-af81-c2ea2bec2585 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.653559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14a8fc-e4f9-488c-9ddc-d7c7c77396f8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.683829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.693314 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.694000 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.694288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.694420 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.694587 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.695312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:32.695439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.695627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.695820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.696093 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.697034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.697410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.698853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.701410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 524/1063] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:32.707535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.712155 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 540/1064] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:32.717905 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:32.729990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:32.827761 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2d8f4-5055-4270-a0bb-0f0eb7f5a22d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.835409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca33052f-68be-4450-a08b-a435bdb3187d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:32.870687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.871368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.871663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.872525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.873350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=ab377869d3a04c58ba0d5798170a8c07, user.id=7078527adc134853a8cf35336771532a) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:32.873673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.875186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.885096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:32.885920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:32.886212 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:32.886497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] PATH_INFO: `/v3/users/nova` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:32.892653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 525/1065] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/role_assignments?user.id=7078527adc134853a8cf35336771532a&scope.project.id=ab377869d3a04c58ba0d5798170a8c07&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:32.895624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:32.897186 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:32.924596 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:32.926360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:32.927942 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:32.930964 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 541/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:32.932383 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.045531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:33.072000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edca77-c9bb-42a3-aa45-4434c424ef5a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:33.112713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:33.113407 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:33.113701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:33.113984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:33.114839 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:33.115161 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:33.117953 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:33.139578 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 526/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:26:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:33.145299 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.202863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:33.259576 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7f3bc-779e-4571-b6d1-c83e6eb0490d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:33.299950 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:33.300676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:33.300962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:33.301239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:33.313046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:33.314497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:33.334191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:33.335363 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:33.336912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:33.336912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:33.336912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:33.336912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:33.340327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 542/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:33.342047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.450332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176e92f-97ac-4986-bbbb-6e995751e512 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:33.486184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:33.486872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:33.487158 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:33.487452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:33.488173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:33.488510 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:33.489908 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:33.500979 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:33.508577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 527/1069] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:33.510904 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.623440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f7ef3-420d-4d02-a971-4d84c0801fd4 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:33.625270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:33.663433 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:33.664240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:33.664557 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:33.664838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:33.672859 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:33.674328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:33.688695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:33.690109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:33.691171 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:33.692362 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 543/1070] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:33.696445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.803992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2f275-5cb9-42c9-b6e2-a0302d601074 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:33.838972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:33.839685 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:33.839975 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:33.840255 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:33.841604 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:33.841923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:33.843360 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0c9c-98a0-47d8-ba9d-072442efd69d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:33.870555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 528/1071] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:33.871790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:33.982210 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6044f1-afab-46fc-95b2-db400056cd4d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:34.021851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:34.022742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:34.023026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:34.023304 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/c6959c9d51794589977e42a360611102` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:34.025549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=d4759f19e262476a8c4c4a99b43daa13, role_id=c6959c9d51794589977e42a360611102)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:34.026980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6306d-8e59-4d31-89e6-def02381701e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:34.042732 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 544/1072] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:26:33 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/d4759f19e262476a8c4c4a99b43daa13/roles/c6959c9d51794589977e42a360611102 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:26:34.157580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0cdc-7d06-41b2-ba60-e291e4142958 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:34.158154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0cdc-7d06-41b2-ba60-e291e4142958 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:34.158154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0cdc-7d06-41b2-ba60-e291e4142958 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:34.159687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 529/1073] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:34.165949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:34.166294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:34.166631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:34.186016 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:34.187084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:34.206850 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:34.273256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 545/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:34.297204 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0cdc-7d06-41b2-ba60-e291e4142958 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:34.365332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:34.412525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0cdc-7d06-41b2-ba60-e291e4142958 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:34.451835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:34.452516 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:34.452805 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:34.453081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:34.526011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:34.527410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:34.593621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:34.645308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 530/1075] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:34.646767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:34.769150 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d66f8-b3bb-4238-b4ed-f7df51351fd3 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:34.815460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:34.816191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:34.816516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:34.816809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:34.827836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:34.829330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-72723418-3100-452f-abd8-c7bd3fa6ac39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:34.849894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:34.851572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:34.852698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:34.852698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:34.852698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:34.852698 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:34.855219 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 546/1076] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:34.857024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:34.969027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ec5a8-f410-405c-a6d8-151f126f1a96 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:35.006925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:35.007601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.007887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.008165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.008926 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:35.009238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:35.010627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216d4ff-b815-405c-a282-7601d0b9da16 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:35.028963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 531/1077] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:35.263917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ee2-ab55-4238-90b9-ac129d8482c1 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.264590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ee2-ab55-4238-90b9-ac129d8482c1 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.264683 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86ee2-ab55-4238-90b9-ac129d8482c1 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.266497 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 547/1078] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:35.272332 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.273105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.273105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.290976 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:35.292184 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:35.309305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:35.381076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 532/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:35.403810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86ee2-ab55-4238-90b9-ac129d8482c1 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:35.408035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:35.515380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86ee2-ab55-4238-90b9-ac129d8482c1 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:35.552835 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:35.553640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.553931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.554212 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.565879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:35.574029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:35.575326 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:35.576351 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 548/1080] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:26:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:35.581961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:35.686983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18bb3f-de04-4dd4-b7fd-8b3f2e7edaac None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:35.722092 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:35.722770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.723058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.723405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.724208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:35.724570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:35.725980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:35.740912 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 533/1081] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:26:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:35.742092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:35.806578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:35.861703 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed906f1-5459-4588-9bd2-aef2987f2989 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:35.897663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:35.898543 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:35.898829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:35.899110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] PATH_INFO: `/v3/users/7078527adc134853a8cf35336771532a` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:35.900330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] RBAC: Authorizing `identity:get_user(user_id=7078527adc134853a8cf35336771532a)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:35.901794 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:35.904118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 549/1082] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:35 2022] GET /identity/v3/users/7078527adc134853a8cf35336771532a => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:35.909789 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:35.925641 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:36.018830 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af440d-b628-4cd1-b059-45db580e26cc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:36.054338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:36.055245 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.055545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.055856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.057138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:36.058552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86834154-2d11-4ade-a691-ed282ee00d01 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:36.061306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 534/1083] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:35 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:36.067488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:36.178674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b11eb-3b3f-400a-8546-62a25b77f94c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:36.211458 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:36.217633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:36.218626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.219079 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.219235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07/users/7078527adc134853a8cf35336771532a/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.221160 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab377869d3a04c58ba0d5798170a8c07, user_id=7078527adc134853a8cf35336771532a, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:36.222584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c70f5-fae4-42ce-bfac-e8220984f657 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:36.237066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 550/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:26:36 2022] PUT /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07/users/7078527adc134853a8cf35336771532a/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:26:36.363236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286d24c-3991-407d-b376-5b9f13f4eb77 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.363795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286d24c-3991-407d-b376-5b9f13f4eb77 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.363795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286d24c-3991-407d-b376-5b9f13f4eb77 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.365168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 535/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:36.371025 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.371589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.371695 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.390197 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:36.391448 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:36.474029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 551/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:36.496820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286d24c-3991-407d-b376-5b9f13f4eb77 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:36.584772 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:36.607541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286d24c-3991-407d-b376-5b9f13f4eb77 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:36.645230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:36.646052 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.646342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.646618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.660193 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:36.669149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:36.671053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:36.672618 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 536/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Thu May 19 20:26:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:36.677673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:36.809396 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7121-f928-4d99-917e-5f34e5fe51f6 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:36.835571 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:36.848613 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:36.849298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:36.850122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:36.850122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:36.850607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:36.850918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:36.852289 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:36.862539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 552/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:26:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:36.868301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:36.975207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34eeb9-fddd-4906-a9da-f6a17b213ebb None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:37.010880 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:37.011681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:37.012024 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:37.012308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:37.023365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:37.026149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:37.027674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:37.048435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:37.049716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:37.051182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:37.051182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:37.051182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:37.051182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:37.051182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 537/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:37.059071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:37.166354 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d0220-f01d-4a33-9030-48315841c414 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:37.202503 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:37.203180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:37.203470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:37.203751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:37.204509 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:37.204823 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:37.206188 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:37.219345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 553/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:37.225039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:37.336431 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd06c-ee5a-452c-ba71-8a42d48e8a04 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:37.379116 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:37.379944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:37.380233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:37.380550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:37.390652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:37.392497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:26:37.408640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:37.410028 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:26:37.411178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 538/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Thu May 19 20:26:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:37.415949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:37.466654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:37.533165 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5717d-91bc-402f-a418-b6f9d551dc97 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:37.571876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:37.572586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:37.572879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:37.573164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:37.574022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:37.574334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:37.576660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:37.594642 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 554/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Thu May 19 20:26:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:37.600795 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:37.667496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:37.727089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f204a-6594-4b25-bd62-fda1ae872c40 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:37.764008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:37.764784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:37.765078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:37.765364 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:37.846934 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:37.848674 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:37.909034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:37.965768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 539/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:37.967134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.071357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915001a-fcf7-4bc2-b9cd-13f3fd2dfe19 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.073322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:38.109839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.110558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.110847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.111130 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.122958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:38.124545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:26:38.143163 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:38.144437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:38.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:26:38.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:26:38.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:26:38.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:38.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 555/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:38.150208 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.255850 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878545c-f6fd-41db-ae25-28e1790b9f8a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.297590 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.298261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.298550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.298835 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.299566 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:38.299878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:38.301306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:38.314446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 540/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:38.322344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.453101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85604011-05df-4326-888a-31b3a14c1e8e None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.489981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.491169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.491458 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.491790 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.499642 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:38.501183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-0e98300d-1068-4d4f-a044-05b9145b46e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:26:38.519327 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:38.520594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:38.521727 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 556/1096] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:26:38.527891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.546405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:38.548920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7c0e0-a760-4b83-a355-5d0554f51deb None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.549257 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7c0e0-a760-4b83-a355-5d0554f51deb None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.549581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7c0e0-a760-4b83-a355-5d0554f51deb None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.550983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 557/1097] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:38.556307 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.556640 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.556989 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.577730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:38.579297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:38.592540 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:38.643516 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32fbb-d5a4-419c-b0b9-7d6d85dcd434 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.654752 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 558/1098] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:26:38.678852 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.679887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.680603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.681132 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.681418 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.683071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:38.683390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:38.685080 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:38.700841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 541/1099] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:38.707139 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.782397 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:38.787655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96288c-f964-45aa-a38e-2b6e08bd176f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.816320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad21b2-fade-455a-89c0-298e1ecc9ac5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:38.829095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.829893 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.830180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.830461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.840958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:26:38.849452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:26:38.850837 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:26:38.852128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 559/1100] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:26:38.859650 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:38.861245 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:38.861923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:38.862216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:38.862622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:38.863366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] RBAC: Adding query filter params (role.id=c6959c9d51794589977e42a360611102, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=d4759f19e262476a8c4c4a99b43daa13) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:38.863720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:38.864611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:38.865216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:38.872025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 542/1101] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/role_assignments?user.id=d4759f19e262476a8c4c4a99b43daa13&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=c6959c9d51794589977e42a360611102 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:26:38.990720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb5ded-9eb8-4ea3-acae-5c6dacf49278 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:39.029180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:39.029895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:39.030184 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:39.030470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:39.031249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:39.031569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:39.033002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:39.047849 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 560/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:26:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:39.049526 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:39.162417 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc324c-71f0-4ae4-8ea2-05fe49bff5f9 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:39.202688 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:39.203585 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:39.203869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:39.204146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] PATH_INFO: `/v3/users/7078527adc134853a8cf35336771532a` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:39.205406 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] RBAC: Authorizing `identity:get_user(user_id=7078527adc134853a8cf35336771532a)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:39.206856 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:39.208827 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 543/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:39 2022] GET /identity/v3/users/7078527adc134853a8cf35336771532a => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:39.223185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:39.296970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:39.341670 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605c350-4d49-4421-9f50-1c340d27d53a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:39.382480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:39.383154 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:39.383447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:39.383734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:39.459965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:39.461569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:39.534855 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:39.573828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 561/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:39.581154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:39.585369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:39.695111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec6dd1-81b3-412b-a1bf-a4f0e91b27de None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:39.732853 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:39.733095 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:39.733221 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:39.733475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:39.734813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:39.736273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292fb35-18ec-4470-bda1-097617792765 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:39.738286 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 544/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:39 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:39.744682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:39.874733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef72f-e383-40e7-9cbe-16923d5be1f3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:39.918352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:39.919209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:39.919594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:39.919884 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:39.920765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=ab377869d3a04c58ba0d5798170a8c07, user.id=7078527adc134853a8cf35336771532a) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:39.921088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:39.922525 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc08aa-f7f4-4f21-9a2e-37601b160b8a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:39.928418 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 562/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:39 2022] GET /identity/v3/role_assignments?user.id=7078527adc134853a8cf35336771532a&scope.project.id=ab377869d3a04c58ba0d5798170a8c07&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:42.236207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f87ce-5234-4f70-96fb-cfa040e334d4 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:42.236797 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f87ce-5234-4f70-96fb-cfa040e334d4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:42.236906 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f87ce-5234-4f70-96fb-cfa040e334d4 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:42.238269 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 545/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:42.244053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:42.244633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:42.244633 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:42.263412 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:42.264651 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:42.287847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:42.335479 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 563/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:42.361606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f87ce-5234-4f70-96fb-cfa040e334d4 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:42.434699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:42.472450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f87ce-5234-4f70-96fb-cfa040e334d4 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:42.509813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:42.510537 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:42.510819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:42.511098 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:42.512337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:42.514150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de1cfc-3c08-4794-94fb-403e6a841bd0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:42.515857 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 546/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:42.522934 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:42.626980 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fa365-aa77-45ca-9084-7fcc84ecdda6 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:42.661988 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:42.662682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:42.662976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:42.663260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:42.664018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:42.665517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d03295-d14c-44d8-9e67-e5f581f81e19 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:42.686316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 564/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:26:42 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:44.978178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275178b-3479-4e84-9313-965643080eaf None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:44.978750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275178b-3479-4e84-9313-965643080eaf None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:44.978750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275178b-3479-4e84-9313-965643080eaf None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:44.980143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 547/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:44.986169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:44.986631 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:44.986736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:45.003662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:45.004948 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:45.016868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:45.075995 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 565/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:45.100337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275178b-3479-4e84-9313-965643080eaf None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:45.206925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275178b-3479-4e84-9313-965643080eaf None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:45.244219 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:45.245187 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:45.245480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:45.245767 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] PATH_INFO: `/v3/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:45.247925 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] RBAC: Authorizing `identity:get_role(role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:45.249350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:45.250768 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 548/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:45 2022] GET /identity/v3/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:45.256739 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:45.360511 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d1370-5997-40a8-9df0-f9ee8a97b6b2 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:45.396249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:45.397152 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:45.397443 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:45.397722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] PATH_INFO: `/v3/users/f5d914b552e142f8a377daee2ceabda1` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:45.411156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d914b552e142f8a377daee2ceabda1)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:45.412623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:45.414315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 566/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:45 2022] GET /identity/v3/users/f5d914b552e142f8a377daee2ceabda1 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:45.425109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:45.437524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:45.532716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e6270-62ff-42ff-a160-064feee2ebf8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:45.569815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:45.570468 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:45.570761 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:45.571046 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:45.644852 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:45.646336 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:45.656984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:45.759983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 549/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:45.766206 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:45.770015 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:45.897943 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46006b-68c0-4ce9-b0c8-1238ce5b9872 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:45.933439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:45.934344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:45.934632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:45.934916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:45.936132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:45.937602 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f7402-e575-4591-a7b9-d72d95dee31a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:45.939462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 567/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:45 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:45.946452 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:46.060561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b632c-a3ab-4b8f-9637-1058e4c3d0d4 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:46.070514 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:46.097345 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:46.098022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:46.098303 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:46.098592 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:46.099442 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=ab377869d3a04c58ba0d5798170a8c07, user.id=f5d914b552e142f8a377daee2ceabda1) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:46.099761 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:46.101283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d81078-17ea-4347-bce4-7f7a721057d4 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:46.108200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 550/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:45 2022] GET /identity/v3/role_assignments?user.id=f5d914b552e142f8a377daee2ceabda1&scope.project.id=ab377869d3a04c58ba0d5798170a8c07&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:48.363316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08a32-0026-476a-aa04-dfd066e64247 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:48.363910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08a32-0026-476a-aa04-dfd066e64247 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:48.363910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef08a32-0026-476a-aa04-dfd066e64247 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:48.365284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 568/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:48.370460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:48.371029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:48.371029 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:48.388424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:48.389601 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:48.459574 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 551/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:48.482836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef08a32-0026-476a-aa04-dfd066e64247 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:48.559143 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:48.590971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef08a32-0026-476a-aa04-dfd066e64247 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:48.628333 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:48.629148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:48.629432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:48.629710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] PATH_INFO: `/v3/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:48.631214 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] RBAC: Authorizing `identity:get_role(role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:48.632625 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:48.634034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 569/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:48 2022] GET /identity/v3/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:48.640121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:48.754222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570a23-caff-4bff-8c11-8e07d827f2f2 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:48.755837 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:48.790837 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:48.791710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:48.791997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:48.792275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] PATH_INFO: `/v3/users/f5d914b552e142f8a377daee2ceabda1` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:48.794266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d914b552e142f8a377daee2ceabda1)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:48.795587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:48.797310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 552/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:48 2022] GET /identity/v3/users/f5d914b552e142f8a377daee2ceabda1 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:48.803468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:48.908730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c8be7-e3f5-4be2-a4bd-682372305002 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:48.947274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:48.948139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:48.948450 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:48.948735 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:48.950620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:48.952273 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5b1da-47dc-4733-8e84-a7e7f6113349 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:48.954037 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 570/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:48 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:48.960014 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:49.067963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698cc16-f9e9-4fc9-90ea-66fd16095927 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:49.103777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:49.104704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:49.104992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:49.105272 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07/users/f5d914b552e142f8a377daee2ceabda1/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:49.108374 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] RBAC: Authorizing `identity:create_grant(project_id=ab377869d3a04c58ba0d5798170a8c07, user_id=f5d914b552e142f8a377daee2ceabda1, role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:49.109850 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782bda1-e0f1-4ae0-9d81-efb828c17e5d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:49.123477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 553/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:26:48 2022] PUT /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07/users/f5d914b552e142f8a377daee2ceabda1/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:26:51.361655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c839-4490-432a-8d3c-8d5319fc7323 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:51.362330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c839-4490-432a-8d3c-8d5319fc7323 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:51.362330 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7c839-4490-432a-8d3c-8d5319fc7323 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:51.363824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 571/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:51.370071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:51.370673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:51.370673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:51.388643 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:51.389949 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:51.429527 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:51.478359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 554/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:51.502447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7c839-4490-432a-8d3c-8d5319fc7323 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:51.514055 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:51.613699 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7c839-4490-432a-8d3c-8d5319fc7323 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:51.649669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:51.650535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:51.650824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:51.651107 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] PATH_INFO: `/v3/roles/ad833f71fc9349d6ad4943d8feec1b59` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:51.652641 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] RBAC: Authorizing `identity:get_role(role_id=ad833f71fc9349d6ad4943d8feec1b59)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:51.654592 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:51.655993 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 572/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:51 2022] GET /identity/v3/roles/ad833f71fc9349d6ad4943d8feec1b59 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:51.661928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:51.766257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf78175-11c5-4e58-90a2-8cce550e851e None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:51.801565 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:51.802447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:51.802732 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:51.803011 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] PATH_INFO: `/v3/users/f5d914b552e142f8a377daee2ceabda1` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:51.804211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d914b552e142f8a377daee2ceabda1)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:51.805707 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:51.807373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 555/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:26:51 2022] GET /identity/v3/users/f5d914b552e142f8a377daee2ceabda1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:51.813213 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:51.895291 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:51.920252 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b639-b26f-4127-998a-02076069f6b7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:51.955358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:51.956017 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:51.956302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:51.956611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:52.025024 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:52.026468 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:52.104302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:52.128219 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 573/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:26:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:26:52.134167 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:52.172885 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:52.247524 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4059e2d3-e109-4086-bbb6-7f4abc27a200 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:52.284261 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:52.285327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:52.285620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:52.285905 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] PATH_INFO: `/v3/projects/ab377869d3a04c58ba0d5798170a8c07` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:52.287094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] RBAC: Authorizing `identity:get_project(project_id=ab377869d3a04c58ba0d5798170a8c07)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:52.288587 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b25dd0f-ad22-4613-abcd-d333b8fa2075 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:52.290317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 556/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:26:52 2022] GET /identity/v3/projects/ab377869d3a04c58ba0d5798170a8c07 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:52.297315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:52.403917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122593a-db1e-4b7a-b817-79e98e8e14e3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:52.443277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:52.444009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:52.444305 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:52.444623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:52.445472 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] RBAC: Adding query filter params (role.id=ad833f71fc9349d6ad4943d8feec1b59, scope.project.id=ab377869d3a04c58ba0d5798170a8c07, user.id=f5d914b552e142f8a377daee2ceabda1) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:26:52.445796 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:52.447244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f11ea-67c5-411c-831c-4a94e43e1abd None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:52.453891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 574/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:26:52 2022] GET /identity/v3/role_assignments?user.id=f5d914b552e142f8a377daee2ceabda1&scope.project.id=ab377869d3a04c58ba0d5798170a8c07&role.id=ad833f71fc9349d6ad4943d8feec1b59 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:54.773005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad2683-3db0-4b52-ab77-c4aa98d1c24a None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:54.773594 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad2683-3db0-4b52-ab77-c4aa98d1c24a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:54.773594 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad2683-3db0-4b52-ab77-c4aa98d1c24a None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:54.774963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 557/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:54.781211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:54.781679 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:54.781792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:54.799606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:54.800922 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:54.838193 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:54.875072 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 575/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:54.898701 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad2683-3db0-4b52-ab77-c4aa98d1c24a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:54.998359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:55.005913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad2683-3db0-4b52-ab77-c4aa98d1c24a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:55.050913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:55.051628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:55.051919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:55.052200 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:55.053426 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:55.054882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4c6a5-f019-4a23-ab34-dd2ef18d74b7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:55.056570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 558/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:55.063353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:55.164935 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65787c16-da05-4b01-a50f-8f53f13cbb18 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:55.201821 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:55.202492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:55.202784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:55.203068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:55.203797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:55.205281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb096d-1425-489f-aeac-02e96ae92140 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:55.232674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 576/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:26:55 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:26:57.573263 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed12d-09ec-4520-81c6-0b7d4b45c9c4 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:57.573920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed12d-09ec-4520-81c6-0b7d4b45c9c4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:57.573920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ed12d-09ec-4520-81c6-0b7d4b45c9c4 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:57.575444 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 559/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:26:57.581720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:57.582266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:57.582397 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:57.600648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:26:57.601892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:26:57.622694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:26:57.683238 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 577/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:26:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:26:57.708774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed12d-09ec-4520-81c6-0b7d4b45c9c4 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:57.818972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ed12d-09ec-4520-81c6-0b7d4b45c9c4 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:57.857243 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:57.858054 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:57.858342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:57.858622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] PATH_INFO: `/v3/domains/default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:57.859844 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:57.861356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaab65-e791-40bf-9d2d-c0614048e273 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:57.863026 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 560/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:26:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:26:57.868944 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:26:57.976314 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350c35-8834-4e5e-a9e7-d2258dd9eee5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:26:58.013600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:26:58.014297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:26:58.014590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:26:58.014875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:26:58.015623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:26:58.017161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed358bc0-7fcf-4e54-95f1-6d07e31739d5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:26:58.039417 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 578/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:26:57 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:00.349523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6cb7-ba94-4410-9d2d-fe46001d4316 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:00.350110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6cb7-ba94-4410-9d2d-fe46001d4316 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:00.350110 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc6cb7-ba94-4410-9d2d-fe46001d4316 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:00.351451 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 561/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:00.357318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:00.357783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:00.357895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:00.375209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:00.376438 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:00.378441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:00.456087 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 579/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:00.479823 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6cb7-ba94-4410-9d2d-fe46001d4316 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:00.515040 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:00.587477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc6cb7-ba94-4410-9d2d-fe46001d4316 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:00.624988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:00.625687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:00.626045 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:00.626256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:00.636283 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:00.644236 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:00.645811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:00.647465 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 562/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:00.652106 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:00.754334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1569c2-92f6-4c6d-bc1c-6a3dc912cab5 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:00.790419 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:00.791099 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:00.791391 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:00.791677 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:00.792460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:00.792782 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:00.794811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:00.799926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:00.803917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 580/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:00.809918 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:00.908472 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:00.919301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456c72b-b65c-4d5e-9bcb-5f8f0073a0bc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:00.953723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:00.954604 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:00.954890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:00.955181 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] PATH_INFO: `/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:00.970461 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] RBAC: Authorizing `identity:get_user(user_id=c19b6c1a574043d8bf7db73cf67c6e4d)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:00.971955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:00.973776 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 563/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:00 2022] GET /identity/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:00.979519 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:01.087620 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aaaf8b-f5f1-4d31-a3e0-117328aa7ee1 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:01.128407 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:01.129096 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:01.129384 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:01.129664 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:01.196680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:01.199153 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:01.201793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:01.308244 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 581/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:27:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:27:01.314322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:01.418816 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874e260-336a-47d0-9816-91fda95d115e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:01.455319 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:01.456102 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:01.456387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:01.456696 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] PATH_INFO: `/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:01.457872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] RBAC: Authorizing `identity:get_project(project_id=ba507c80ef744a7b9fb2ab43cc5e2de6)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:01.459309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4fc1-e655-4a61-a609-47ef9847254b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:01.461069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 564/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:01 2022] GET /identity/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:01.467259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:01.568356 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:01.571463 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0978-12c9-46bb-9a11-81d29f205b41 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:01.608875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:01.609546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:01.609841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:01.610129 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:01.610978 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=ba507c80ef744a7b9fb2ab43cc5e2de6, user.id=c19b6c1a574043d8bf7db73cf67c6e4d) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:01.611298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:01.612767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa1288-a1f8-45e8-b677-e4aca6b25cda None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:01.618278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 582/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:27:01 2022] GET /identity/v3/role_assignments?user.id=c19b6c1a574043d8bf7db73cf67c6e4d&scope.project.id=ba507c80ef744a7b9fb2ab43cc5e2de6&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:03.878355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ac3af-6070-4449-89ff-bf1092c314ab None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:03.878937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ac3af-6070-4449-89ff-bf1092c314ab None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:03.878937 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ac3af-6070-4449-89ff-bf1092c314ab None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:03.880316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 565/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:03.886405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:03.886959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:03.886959 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:03.904932 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:03.906134 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:03.976227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 583/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:04.003842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ac3af-6070-4449-89ff-bf1092c314ab None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:04.023971 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:04.110352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ac3af-6070-4449-89ff-bf1092c314ab None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:04.146846 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:04.147663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:04.147952 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:04.148233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:04.159544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:04.172581 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:04.173942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:04.175387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 566/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:04.180713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:04.282533 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:04.289799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d920c7-cd2e-4ea6-af88-1ec957cdcdab None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:04.326510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:04.327190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:04.327483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:04.327768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:04.328556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:04.328875 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:04.330255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:04.338380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 584/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:04.344833 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:04.427301 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:04.449276 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8265a8-dac8-4d5f-9f0c-74c798ac26ab None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:04.488892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:04.488892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:04.488892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:04.488892 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] PATH_INFO: `/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:04.489433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] RBAC: Authorizing `identity:get_user(user_id=c19b6c1a574043d8bf7db73cf67c6e4d)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:04.489819 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:04.491497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 567/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:04 2022] GET /identity/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:04.497660 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:04.607164 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd6d00-285a-4928-ae7a-49eab279094d None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:04.642928 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:04.643793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:04.644077 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:04.644354 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] PATH_INFO: `/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:04.645585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ba507c80ef744a7b9fb2ab43cc5e2de6)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:04.647014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036893e5-9c50-45ec-a4de-d75c7006d2f4 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:04.648784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 585/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:04 2022] GET /identity/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:04.656165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:04.772324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac385a17-8f31-4a32-b587-9c43bb951303 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:04.807902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:04.808889 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:04.809293 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:04.809756 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] PATH_INFO: `/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6/users/c19b6c1a574043d8bf7db73cf67c6e4d/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:04.820324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba507c80ef744a7b9fb2ab43cc5e2de6, user_id=c19b6c1a574043d8bf7db73cf67c6e4d, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:04.821866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ecacb-f07c-41cb-9871-2dc29b35d425 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:04.849863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 568/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:27:04 2022] PUT /identity/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6/users/c19b6c1a574043d8bf7db73cf67c6e4d/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:27:07.080370 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f90e4-02ae-41d7-be55-caf4bd924bfa None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.080957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f90e4-02ae-41d7-be55-caf4bd924bfa None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.080957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f90e4-02ae-41d7-be55-caf4bd924bfa None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.082352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 586/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:07.087760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.088216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.088330 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.100202 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:07.107265 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:07.108854 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:07.177877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 569/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:07.202328 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f90e4-02ae-41d7-be55-caf4bd924bfa None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:07.230281 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:07.311509 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f90e4-02ae-41d7-be55-caf4bd924bfa None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:07.348811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:07.349656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.349947 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.350232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.361056 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:07.368912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:07.370321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:27:07.372038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 587/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:07.377009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:07.483876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc72547-1543-4427-ba1a-b457244f79d8 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:07.520328 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:07.521194 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.521369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.521655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.522423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:07.522822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:07.524175 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:07.527338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:07.535485 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 570/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:07.541368 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:07.632361 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:07.649599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ad689-34e7-4bea-a5b1-92daf63907e9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:07.685684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:07.686588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.686879 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.687168 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] PATH_INFO: `/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.688517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] RBAC: Authorizing `identity:get_user(user_id=c19b6c1a574043d8bf7db73cf67c6e4d)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:07.690233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:07.691933 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 588/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:07 2022] GET /identity/v3/users/c19b6c1a574043d8bf7db73cf67c6e4d => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:07.697735 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:07.800165 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b24f6-ff32-41bc-a994-fabab6d49ffd None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:07.834316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:07.834964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:07.835275 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:07.835559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:07.905471 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:07.909623 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:07.911115 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:08.018538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 571/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:27:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:27:08.024480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:08.128765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b97bb-051a-4b42-86c5-34e2b1e037be None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:08.163810 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:08.164615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:08.164899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:08.165191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] PATH_INFO: `/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:08.166377 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] RBAC: Authorizing `identity:get_project(project_id=ba507c80ef744a7b9fb2ab43cc5e2de6)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:08.167873 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60179e-c01d-429b-9d64-b5af6fc84506 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:08.169629 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 589/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:08 2022] GET /identity/v3/projects/ba507c80ef744a7b9fb2ab43cc5e2de6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:08.175729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:08.309866 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:08.316878 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040139b-eebb-48bb-ba73-f4962018b9b6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:08.354424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:08.355073 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:08.355498 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:08.355946 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:08.356874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=ba507c80ef744a7b9fb2ab43cc5e2de6, user.id=c19b6c1a574043d8bf7db73cf67c6e4d) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:08.357214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:08.358671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d39d87-ba17-4e75-8700-5b6dd0cd6ed2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:08.365393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 572/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:27:08 2022] GET /identity/v3/role_assignments?user.id=c19b6c1a574043d8bf7db73cf67c6e4d&scope.project.id=ba507c80ef744a7b9fb2ab43cc5e2de6&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:10.630394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e86f20-7db8-452a-9b49-c095d1368f05 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:10.630970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e86f20-7db8-452a-9b49-c095d1368f05 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:10.630970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e86f20-7db8-452a-9b49-c095d1368f05 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:10.632322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 590/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:10.637840 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:10.638294 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:10.638400 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:10.657343 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:10.658582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:10.728928 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 573/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:27:10.751716 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e86f20-7db8-452a-9b49-c095d1368f05 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:10.775555 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:10.861721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e86f20-7db8-452a-9b49-c095d1368f05 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:10.897938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:10.898603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:10.898891 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:10.899172 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:10.978260 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:10.979728 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:11.001805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:11.083681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 591/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:27:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:27:11.089724 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:11.185404 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:11.194778 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36676506-c3fa-4bc5-9411-025c791af910 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:11.229612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:11.230436 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:11.231356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:11.231644 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:11.239117 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:11.240614 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5ebe293c-37f3-4bdd-b4c3-6ee33bc868aa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:11.255393 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:11.256838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:11.257899 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 574/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:27:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:11.262718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:11.366116 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29139219-3db0-4720-9c34-4ae764abbe04 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:11.388173 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:11.402190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:11.403439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:11.403723 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:11.403999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:11.404762 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:11.405070 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:11.406432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387b822-0075-4f03-aa32-af908f0d1335 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:11.413600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 592/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu May 19 20:27:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:13.665744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d676c-df87-4d84-9214-02419cc384c7 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:13.666391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d676c-df87-4d84-9214-02419cc384c7 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:13.666391 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d676c-df87-4d84-9214-02419cc384c7 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:13.667801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 575/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:13.674647 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02896be-a26b-443e-b8c6-b61a251fcf98 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:13.675207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02896be-a26b-443e-b8c6-b61a251fcf98 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:13.675207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02896be-a26b-443e-b8c6-b61a251fcf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:13.693112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-f02896be-a26b-443e-b8c6-b61a251fcf98 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:13.694324 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-f02896be-a26b-443e-b8c6-b61a251fcf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:13.764923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 593/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:13.790837 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d676c-df87-4d84-9214-02419cc384c7 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:13.899663 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d676c-df87-4d84-9214-02419cc384c7 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:13.934938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:13.935627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:13.935920 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:13.936207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:13.937043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:13.938482 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de472e-6fb6-4e69-8910-a25077eb3787 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:13.955337 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 576/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu May 19 20:27:13 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:27:16.280882 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2555ee-cb84-46f2-9089-4db9361b26ba None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:16.281572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2555ee-cb84-46f2-9089-4db9361b26ba None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:16.281572 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2555ee-cb84-46f2-9089-4db9361b26ba None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:16.282895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 594/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:16.288864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:16.289382 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:16.289492 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:16.306804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:16.308013 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:16.315953 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:16.379890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 577/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:16.404500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2555ee-cb84-46f2-9089-4db9361b26ba None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:16.509321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2555ee-cb84-46f2-9089-4db9361b26ba None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:16.523933 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:16.545174 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:16.545975 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:16.546270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:16.546560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] PATH_INFO: `/v3/domains/0835e562729f4861832d55c5e2d46c01` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:16.557656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0835e562729f4861832d55c5e2d46c01)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:16.559638 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6fbfb1-f397-4df1-b2c6-cb61f2ad7f04 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:16.561369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 595/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 20:27:16 2022] GET /identity/v3/domains/0835e562729f4861832d55c5e2d46c01 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:16.567520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:16.681242 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df936bbe-5eb7-4993-8a9a-3328e0cfc7e3 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:16.726887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:16.727555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:16.727845 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:16.728155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:16.728914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:16.730373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534af5e-54c4-4be0-83f9-9f0c514489da None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:16.743750 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:16.758322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 578/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:27:16 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:19.077930 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2daa1-1e42-4acd-9f50-b1a5e46da3bf None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:19.078507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2daa1-1e42-4acd-9f50-b1a5e46da3bf None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:19.078507 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2daa1-1e42-4acd-9f50-b1a5e46da3bf None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:19.079876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 596/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:19.085387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:19.085828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:19.085954 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:19.104489 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:19.106155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:19.176142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 579/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:19.202590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2daa1-1e42-4acd-9f50-b1a5e46da3bf None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:19.307839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2daa1-1e42-4acd-9f50-b1a5e46da3bf None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:19.345503 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:19.346378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:19.346669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:19.346956 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] PATH_INFO: `/v3/domains/0835e562729f4861832d55c5e2d46c01` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:19.348152 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=0835e562729f4861832d55c5e2d46c01)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:19.349630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923021d2-7c11-4491-87fc-f7be5f592e0e None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:19.351459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 597/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu May 19 20:27:19 2022] GET /identity/v3/domains/0835e562729f4861832d55c5e2d46c01 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:19.358083 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:19.467023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85eae9-baef-4d37-96d6-21f5d5423582 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:19.492771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:19.505362 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:19.506048 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:19.506340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:19.506627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:19.507358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:19.508984 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471bb3d-7466-44b3-a646-67c629da8bdb None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:19.530659 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 580/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:27:19 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:27:21.856516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feaa9-4f90-4080-aef9-b1535f9fbb2f None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:21.857046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feaa9-4f90-4080-aef9-b1535f9fbb2f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:21.857161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707feaa9-4f90-4080-aef9-b1535f9fbb2f None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:21.859112 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 598/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:21.865038 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:21.865599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:21.865599 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:21.883134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:21.884340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:21.955947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 581/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:21.980240 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707feaa9-4f90-4080-aef9-b1535f9fbb2f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:22.034793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:22.094439 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707feaa9-4f90-4080-aef9-b1535f9fbb2f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:22.129177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:22.129901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:22.130216 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:22.130447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:22.141259 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:22.148971 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:22.150427 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:27:22.151700 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 599/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:22.156977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:22.257355 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24546a3f-f640-436b-a929-3e698d9a1a9a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:22.268770 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:22.295511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:22.296185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:22.296500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:22.296785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:22.297577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:22.297847 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:22.299230 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:22.307439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 582/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:22.314585 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:22.433230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb838ced-8e1c-4088-8a0e-aa1f20f6be9a None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:22.435778 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:22.471654 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:22.472560 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:22.472854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:22.473139 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] PATH_INFO: `/v3/users/c1da3c887e464a8b860529458717a6a6` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:22.497051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] RBAC: Authorizing `identity:get_user(user_id=c1da3c887e464a8b860529458717a6a6)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:22.498558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:22.500358 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 600/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:22 2022] GET /identity/v3/users/c1da3c887e464a8b860529458717a6a6 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:22.506213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:22.613428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48577d3-2ad7-4f72-9e83-50f6f5ab421d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:22.655229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:22.655900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:22.656190 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:22.656509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:22.701528 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:22.729215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:22.731438 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:22.845525 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 583/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:27:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:27:22.851746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:22.957937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d59bfc-2f4a-48da-bdac-bb6f491b0823 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:22.993469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:22.994333 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:22.994627 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:22.994909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] PATH_INFO: `/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:22.996158 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1cd5cc88b14df4ac9a695fcce511ce)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:22.997639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087f81f-197b-470d-8720-9410c49bd699 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:22.999371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 601/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:22 2022] GET /identity/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:23.005698 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:23.077738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:23.108366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e0aaf-18ad-4528-8af1-8c7693a77796 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:23.145542 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:23.146195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:23.146481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:23.146759 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:23.147596 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=2a1cd5cc88b14df4ac9a695fcce511ce, user.id=c1da3c887e464a8b860529458717a6a6) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:23.147902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:23.149395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab7e90-ea51-4442-8ce4-bcfbed5a5569 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:23.154921 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 584/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:27:23 2022] GET /identity/v3/role_assignments?user.id=c1da3c887e464a8b860529458717a6a6&scope.project.id=2a1cd5cc88b14df4ac9a695fcce511ce&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:25.398038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf49b-069f-4eea-abd8-733f868a437d None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:25.398604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf49b-069f-4eea-abd8-733f868a437d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:25.398604 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bf49b-069f-4eea-abd8-733f868a437d None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:25.400006 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 602/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:25.406667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:25.407222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:25.407333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:25.424671 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:25.425876 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:25.495542 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 585/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:25.524539 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf49b-069f-4eea-abd8-733f868a437d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:25.554271 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:25.633736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bf49b-069f-4eea-abd8-733f868a437d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:25.671247 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:25.672301 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:25.672623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:25.672901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:25.685264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:25.693345 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:25.694684 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:27:25.695786 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 603/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:25.701464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:25.768602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:25.804869 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0ae0f-62e5-4968-b493-c60c8def4dda None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:25.840499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:25.841173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:25.841463 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:25.841745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:25.842495 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:25.842809 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:25.844197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:25.852545 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 586/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:25.859038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:25.968287 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:25.984333 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bb1c2-82a1-47fb-a5c4-afe991bf4c44 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:26.018741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:26.020251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:26.020792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:26.021196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] PATH_INFO: `/v3/users/c1da3c887e464a8b860529458717a6a6` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:26.023031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=c1da3c887e464a8b860529458717a6a6)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:26.025241 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:26.028277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 604/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:25 2022] GET /identity/v3/users/c1da3c887e464a8b860529458717a6a6 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:26.034225 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:26.137955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d5517-7517-42fb-a447-a81e9e676bcc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:26.164439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:26.172969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:26.173834 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:26.174121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:26.174403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] PATH_INFO: `/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:26.175700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] RBAC: Authorizing `identity:get_project(project_id=2a1cd5cc88b14df4ac9a695fcce511ce)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:26.177227 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc22c9-99bd-4d18-90f4-0f5e5ef5053e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:26.179027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 587/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:26 2022] GET /identity/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:26.185227 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:26.287420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a140bfd-7507-4001-a087-a00a6219a1e8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:26.322623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:26.323511 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:26.323793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:26.324097 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] PATH_INFO: `/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce/users/c1da3c887e464a8b860529458717a6a6/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:26.326249 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1cd5cc88b14df4ac9a695fcce511ce, user_id=c1da3c887e464a8b860529458717a6a6, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:26.327705 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb254ff4-ead5-4e3d-a695-017c094933e1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:26.343658 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 605/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:27:26 2022] PUT /identity/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce/users/c1da3c887e464a8b860529458717a6a6/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:27:28.621256 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c4f4-901b-40ba-ac84-dcba6c6d53c0 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:28.621841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c4f4-901b-40ba-ac84-dcba6c6d53c0 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:28.621841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c4f4-901b-40ba-ac84-dcba6c6d53c0 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:28.623228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 588/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:28.629603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:28.630161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:28.630410 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:28.650179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:28.651437 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:28.667767 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:28.741784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 606/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:27:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:27:28.780154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c4f4-901b-40ba-ac84-dcba6c6d53c0 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:28.896704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c4f4-901b-40ba-ac84-dcba6c6d53c0 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:28.933488 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:28.934496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:28.934781 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:28.935064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:28.946327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:28.954944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:28.956376 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:27:28.957939 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 589/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:27:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:28.962559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:29.071251 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c38d17-7f58-40e2-9a8b-b1950a81b861 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:29.106811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:29.107500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:29.107787 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:29.108069 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:29.108917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:29.109233 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:29.110772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:29.119966 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 607/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:27:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:29.126718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:29.201437 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:29.249923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6abc6-8fe2-4664-84b1-5a1b701b7de6 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:29.286757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:29.287639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:29.287924 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:29.288203 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] PATH_INFO: `/v3/users/c1da3c887e464a8b860529458717a6a6` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:29.289449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] RBAC: Authorizing `identity:get_user(user_id=c1da3c887e464a8b860529458717a6a6)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:29.290887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:29.292867 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 590/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu May 19 20:27:29 2022] GET /identity/v3/users/c1da3c887e464a8b860529458717a6a6 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:29.299208 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:29.311343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:29.420513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d678e-4c06-49cf-8b20-c7b91b676cb5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:29.457117 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:29.457690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:29.457981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:29.458264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:29.532609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:29.534061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:29.544653 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:29.654352 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 608/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:27:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:27:29.660459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:29.765320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b58cf-92f7-4987-b660-367af5ab54f3 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:29.790142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:29.802368 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:29.803244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:29.803531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:29.803815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] PATH_INFO: `/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:29.805044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1cd5cc88b14df4ac9a695fcce511ce)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:29.806499 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce487-7870-495b-97ae-2c30a625e255 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:29.808247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 591/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu May 19 20:27:29 2022] GET /identity/v3/projects/2a1cd5cc88b14df4ac9a695fcce511ce => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:29.815674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:29.925071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53612b51-6dbd-4908-8b8a-f6772c019ea8 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:29.935093 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:29.962230 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:29.962898 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:29.963191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:29.963478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:29.964323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=2a1cd5cc88b14df4ac9a695fcce511ce, user.id=c1da3c887e464a8b860529458717a6a6) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:29.965215 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:29.966801 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b9eb6-a740-4ce7-8bb8-53395134e3d6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:29.972714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 609/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:27:29 2022] GET /identity/v3/role_assignments?user.id=c1da3c887e464a8b860529458717a6a6&scope.project.id=2a1cd5cc88b14df4ac9a695fcce511ce&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:39.797919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=4 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.969979 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=14 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.202103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657e3f-d589-4ef0-90f9-d401d8375ec7 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.202726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657e3f-d589-4ef0-90f9-d401d8375ec7 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.202726 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9657e3f-d589-4ef0-90f9-d401d8375ec7 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.204322 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 592/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:30.210519 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.211224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.211224 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.230432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:30.231687 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:30.328530 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 610/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:28:30.352186 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9657e3f-d589-4ef0-90f9-d401d8375ec7 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.474041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9657e3f-d589-4ef0-90f9-d401d8375ec7 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.512008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.512935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.513223 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.513505 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.525088 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.527409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:30.545563 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:30.546803 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:30.548051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:30.548051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:30.548051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:30.548051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:30.548051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 593/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:30.552753 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.685428 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39130a5e-b314-4e3a-9d52-cc31ba0db288 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.719860 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.729503 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.730320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.730609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.730889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.732812 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:30.733142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.734590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46418a1-3c92-4fd1-b2ad-f5a0c2333cb1 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.748332 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 611/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:30.754262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.828076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.895877 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115422e-b48f-46b8-8d3d-e0352da48a46 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.937977 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.938730 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.939025 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.939313 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.940252 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.942063 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca27e2e-e59f-4dc3-8cf9-c6420735af58 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.976309 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 594/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu May 19 20:28:30 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:33.548619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33428d5-de0a-4610-a89b-d888f287ffef None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:33.549320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33428d5-de0a-4610-a89b-d888f287ffef None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:33.549320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33428d5-de0a-4610-a89b-d888f287ffef None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:33.550992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 612/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:33.557595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:33.558209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:33.558467 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:33.578699 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:33.580247 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:33.669743 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 595/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:28:33.696105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33428d5-de0a-4610-a89b-d888f287ffef None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:33.852851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33428d5-de0a-4610-a89b-d888f287ffef None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:33.904006 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:33.905004 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:33.905298 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:33.905581 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:33.919142 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:33.927918 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:33.929381 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:33.930535 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 613/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:28:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:33.937645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:33.986036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.067273 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6c94-a106-44a4-9cf8-5090d57955fd None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.119396 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.120907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.121224 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.121570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.122500 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:34.122935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.124556 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.135846 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 596/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:28:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:34.141809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.215323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.282157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68cb06-9e8d-4005-8bb6-0b3a78b801f0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.337444 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.338543 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.338838 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.339128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.353524 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.355492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:34.377108 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:34.378589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:34.379671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:34.379671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:34.379671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:34.379671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:34.379671 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 614/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:34.387232 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.540162 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.543919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ff1de-b44d-44ed-9acb-1b957109f791 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.584180 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.585037 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.585215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.585411 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.586114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:34.586428 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.587820 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.611430 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 597/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:34.611668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.746817 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eb870-a309-4700-9149-5a43149367b3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.748960 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.805561 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.806388 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.806676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.806977 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.820459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.821667 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:34.840159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:34.841649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:34.843027 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:34.850655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.851661 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 615/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:34.973602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0009-b237-4468-b3f2-19000a10bce7 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.021635 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.022757 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.023069 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.023416 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.024626 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:35.025012 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.027365 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.054872 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 598/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.058447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.188009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba97a5-8510-481c-945a-c6f1b8c3dd87 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.228495 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.229237 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.229475 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.229755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.283297 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.342375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.353429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.489765 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 616/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:28:35.496649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.520595 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.626021 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95305a-83ab-46e5-b490-d79710ccdd6f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.681143 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.682441 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.682652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.683081 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.697076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.701695 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:35.723879 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:35.725105 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:35.726199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:35.726199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:35.726199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:35.726199 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:35.730022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 599/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:35.732579 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.811889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.864266 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33230034-80be-4083-970e-ccd0d3db3748 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.910681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.911366 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.911662 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.911945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.912725 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:35.913045 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.914429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.928284 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 617/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.934961 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.023799 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.071039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890ee26-6013-47cb-872b-0b1e2542222d None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.115254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.116130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.116456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.116747 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.125195 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.127169 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:36.145342 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:36.146603 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:36.147907 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:36.151662 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 600/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:36.152793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.282308 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcebd92-fce2-4619-985f-2def06497d92 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.326246 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.350976 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.351751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.352060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.352331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.353199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:36.353509 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.355759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e249e64-164c-4f2d-aee6-c4ce2b15bafb None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.370190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 618/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:36.376725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.505675 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f416352c-ba52-4b0e-8e36-84db4ec515a2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.541717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.555870 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.556621 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.556942 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.557785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.558605 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=850ca515e47a4d0195bba527a2a1696f) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:36.558964 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.560645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2810b-b267-4a9d-b207-cd2ac297a50b None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.567144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 601/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/role_assignments?user.id=850ca515e47a4d0195bba527a2a1696f&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.053851 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea447299-dbf8-4b14-88c0-e8ecdf1fc13f None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.054483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea447299-dbf8-4b14-88c0-e8ecdf1fc13f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.054483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea447299-dbf8-4b14-88c0-e8ecdf1fc13f None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.055808 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 619/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:39.062874 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.063511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.063511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.080995 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:39.082210 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:39.162831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 602/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:28:39.185584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea447299-dbf8-4b14-88c0-e8ecdf1fc13f None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.308420 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea447299-dbf8-4b14-88c0-e8ecdf1fc13f None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.350028 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.350938 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.351320 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.351600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.372517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:39.380617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:39.382014 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:39.383348 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 620/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:39.388919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.504863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1b98d-effb-42cc-b52b-98bf631fd165 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.532375 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.544433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.545129 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.545424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.545692 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.546476 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] RBAC: Adding query filter params (name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:39.546762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.548142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.563593 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 603/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.565229 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.617803 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.699449 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b062e3-79b2-4427-ad6e-b2d74a661848 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.737805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.738706 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.739083 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.739417 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.751965 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.753606 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:39.773567 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:39.774772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:39.776569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:39.776569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:39.776569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:39.776569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:39.776569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 621/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:39.781559 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.903125 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010075db-43ef-46e5-aa71-bc46fdb1749e None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.953217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.953913 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.954207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.954492 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.955231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:39.955550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.956993 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.970582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 604/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.976372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.085026 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.096486 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e5b12-283a-4d1f-af87-7abeef033f69 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.136084 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.137288 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.137669 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.138064 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] PATH_INFO: `/v3/users/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.148874 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.150387 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:40.168908 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:40.170319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:40.171600 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 622/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:40.176144 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.236660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.298207 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d6e00-af27-4a67-8da8-1767891c75ea None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.339338 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.339933 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.340241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.340553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.341410 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:40.341851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.343198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.363265 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 605/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:40.369229 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.482445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d41e2-3bbf-4eb1-b4d9-c4773daf9b0b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.522253 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.523075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.523425 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.523793 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.538733 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.540235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:40.558878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:40.560379 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:40.561836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:40.561836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:40.561836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:40.561836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:40.561836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 623/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:40.566785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.682538 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5dae5-d194-4e76-b8eb-d7119443b533 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.685441 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.729691 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.730395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.730681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.730959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.731703 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:40.732008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.733477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.745723 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 606/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:40.751492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.762098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.877902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4f361-532e-41f7-8264-781fa313c907 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.916458 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.917334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.917656 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.917926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.925273 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.926721 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:40.943573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:40.945264 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:40.946708 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 624/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:40.951686 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.071600 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845b025-06f0-4b23-8ec9-7a5d712a3411 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.124060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.124773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.125067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.125352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.126219 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:41.126622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.128264 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dc8ea-cca1-456e-8735-2356b6beb0cd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.151550 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 607/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:41.157630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.230104 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:41.269857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71802e-1d01-420a-9a3c-7dd3cb2738e9 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.310369 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.311344 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] REQUEST_METHOD: `PUT` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.311655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.311970 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/850ca515e47a4d0195bba527a2a1696f/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.330009 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=850ca515e47a4d0195bba527a2a1696f, role_id=2f3fc3dd4a0c4a73bd23497cdf9a8031)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.331755 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec52692-2a0d-4575-a50e-ddb5f9858175 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.348841 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 625/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:28:41 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/850ca515e47a4d0195bba527a2a1696f/roles/2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:43.763022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1f48-4ab3-4bc2-a245-4463e4d0c9c5 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.763572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1f48-4ab3-4bc2-a245-4463e4d0c9c5 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.763572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1f48-4ab3-4bc2-a245-4463e4d0c9c5 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.765061 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 608/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:43.771166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.771655 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.771809 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.793500 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:43.795441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:43.877589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 626/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:28:43.902555 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec1f48-4ab3-4bc2-a245-4463e4d0c9c5 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.930148 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.039147 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec1f48-4ab3-4bc2-a245-4463e4d0c9c5 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.089715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.090529 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.090814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.091111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.103122 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:44.113150 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:44.114490 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:44.115955 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 609/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:44.121115 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.172704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.233409 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb61d1b-aba9-4400-8c65-691fb999c220 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.272676 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.273355 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.273643 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.273926 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.274736 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] RBAC: Adding query filter params (name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:44.274996 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.277239 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.285813 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 627/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:44.292197 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.398655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.420814 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11250a8-bece-4b36-aa83-b90240eeae40 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.458350 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.459161 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.459450 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.459729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.471764 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.473289 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:44.492718 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:44.494087 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:44.495983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:44.495983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:44.495983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:44.495983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:44.495983 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 610/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:44.500360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.613836 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020a719-ce55-46b3-ad99-a8b4fb8401bf None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.624242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.655429 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.656636 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.657013 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.657404 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.658221 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:44.658601 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.660083 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.677371 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 628/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:44.683781 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.801716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3a8aa-0775-4c06-ba99-02c6c297e114 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.840801 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.841667 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.841958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.842239 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.845306 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.852867 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.855257 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:44.876089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:44.877552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:44.878727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 611/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:44.883870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.014325 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66561fe2-bd43-4eca-be9d-6e6878fdb67b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.052704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.053383 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.053672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.053952 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.054804 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:45.055118 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.056584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.081078 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 629/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:45.087483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.218244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e96c3b-4f92-40e4-a294-a9edb2fb9295 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.259636 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.260981 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.261335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.261522 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.348536 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.358241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.359745 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.478731 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 612/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:28:45.484675 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.517207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.599720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a12693-22cb-48e7-81a5-91ded911b0d0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.640199 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.641039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.641322 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.641611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.652834 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.654319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:45.674999 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:45.676290 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:45.677580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:45.677580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:45.677580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:45.677580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:45.677580 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 630/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:45.682513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.794390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.801178 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fd25-0017-43cc-9f18-0e4a4d4b83b5 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.848523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.849222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.849507 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.849785 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.850616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:45.851041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.852744 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.868446 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 613/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:45.874666 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.965121 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.989630 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c374c-4e3b-4a14-85c6-b90e622d6ec5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.031066 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.032042 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.032225 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.032549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.041682 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.043609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:46.060074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:46.061395 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:46.062559 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 631/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:46.067832 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.196580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d6f94-dada-44c0-8e15-36f867fb7adc None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.236085 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.236956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.237405 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.237766 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.238721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:46.239124 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.240670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5e25d-e500-4e3a-932f-449dd06c95ff None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.254838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 614/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:46.261595 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.391750 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6eabae-640c-428a-b77e-26978fc49488 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.434707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.435578 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.435741 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.436059 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.437035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] RBAC: Adding query filter params (role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=850ca515e47a4d0195bba527a2a1696f) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:46.437376 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.438953 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93d3d7-8ba7-4486-b471-cdaaf9d7cc94 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.450357 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 632/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/role_assignments?user.id=850ca515e47a4d0195bba527a2a1696f&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=2f3fc3dd4a0c4a73bd23497cdf9a8031 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:48.810042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bb01c-9698-4d73-b641-61d62f7f173b None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.810649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bb01c-9698-4d73-b641-61d62f7f173b None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.810649 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bb01c-9698-4d73-b641-61d62f7f173b None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.812185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 615/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:48.818226 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.818877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.818877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.824433 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:48.839889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:48.841207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:48.926916 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 633/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:28:48.953215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bb01c-9698-4d73-b641-61d62f7f173b None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:48.973682 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:49.079950 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bb01c-9698-4d73-b641-61d62f7f173b None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:49.121386 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:49.122366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:49.122706 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:49.123042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:49.135456 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:49.145070 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:49.146620 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:49.147842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 616/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:28:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:49.153877 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:49.287086 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92eb09-9c6b-413a-a400-18a3437f54de None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:49.325135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:49.331434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:49.332196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:49.332586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:49.332937 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:49.333770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:49.334154 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:49.335594 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:49.346659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 634/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:28:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:49.352654 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:49.455810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:49.487815 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11a7f6-6a8f-4681-bcd2-1e2e56de2f5a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:49.533191 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:49.534027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:49.534325 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:49.534610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:49.548308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:49.550042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:49.570990 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:49.572602 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:49.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:49.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:49.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:49.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:49.573940 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 617/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:49.578768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:49.698603 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dca13e-540a-4610-a0ea-c48d3493ea0b None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:49.738445 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:49.739414 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:49.739714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:49.740101 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:49.740906 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:49.741527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:49.743085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:49.758030 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 635/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:49.764758 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:49.894478 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1c1e7-350a-4bb9-9894-2f258b3e92ee None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:49.927415 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:49.937403 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:49.938327 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:49.938619 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:49.938902 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:49.950373 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:49.951813 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:49.968958 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:49.970729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:49.972023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 618/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:49.976295 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:50.046319 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:50.099398 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207b6f8-9d52-43c8-8ca7-f5fa0aee85f5 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:50.139061 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:50.139739 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:50.140092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:50.141687 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:50.142550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:50.142863 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:50.144312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:50.164510 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 636/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:50.170935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:50.315849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25aee70-9ec2-4d2e-bb6c-f833259b8ae8 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:50.370808 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:50.371740 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:50.372174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:50.372496 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:50.450233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:50.464135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:50.465760 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:50.594233 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 619/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:28:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:28:50.595461 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:50.721482 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbd31-85d2-435f-aa7c-e0eae67b6be6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:50.738569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:50.763632 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:50.764462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:50.764750 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:50.765029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:50.776430 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:50.777718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:50.797043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:50.803076 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:50.804607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:50.804607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:50.804607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:50.804607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:50.804607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 637/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:50.805026 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:50.910369 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:50.926395 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc123f2-59c6-4748-8c82-7ae6a3f630aa None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:50.970622 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:50.971324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:50.971616 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:50.971900 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:50.972748 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:50.973067 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:50.974558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:50.993003 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 620/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:50.994929 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:51.124060 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048eeb3-bb6c-42d5-8673-e77fcdf9f689 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:51.165573 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:51.166957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.167331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.167518 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.175216 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:51.176894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:51.194967 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:51.196441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:51.198043 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 638/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:51.203185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:51.330044 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9834245-3b8d-41de-8df2-5105bfbdcd96 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:51.380030 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:51.380608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.380895 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.381170 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] PATH_INFO: `/v3/projects` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.381967 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:51.382281 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:51.383754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0f6e7d-4539-407c-81a5-f63433afa081 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:51.398460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 621/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:28:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:51.405668 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:51.433861 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:51.524565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54d034-8612-450f-a5e3-187e500f2898 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:51.564162 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:51.565021 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.565408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.565889 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.567498 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=850ca515e47a4d0195bba527a2a1696f) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:51.567968 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:51.569883 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6707a69-f50c-4396-a614-082f8bca3a2a None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:51.578839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 639/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:28:51 2022] GET /identity/v3/role_assignments?user.id=850ca515e47a4d0195bba527a2a1696f&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:53.978194 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cd2b-5db7-44fe-a03e-53972b68f8ea None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.978784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cd2b-5db7-44fe-a03e-53972b68f8ea None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.978784 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0cd2b-5db7-44fe-a03e-53972b68f8ea None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.980329 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 622/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:53.985649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.986085 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.986220 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.003372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:54.004628 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:54.093764 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 640/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:28:54.116610 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0cd2b-5db7-44fe-a03e-53972b68f8ea None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:54.128539 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.233043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0cd2b-5db7-44fe-a03e-53972b68f8ea None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.270042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.270843 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.271135 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.280152 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.291864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:54.299774 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:54.300997 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:54.302477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 623/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:28:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:54.307772 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:54.327321 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.423745 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81979647-58ea-4a51-9a14-5110252dcc2c None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.459996 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.460773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.461068 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.461353 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] PATH_INFO: `/v3/roles` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.462113 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:54.462431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:54.464051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:54.472182 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 641/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:28:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:54.478062 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:54.547441 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.583053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800482d-1c5f-4393-ae40-da12dd010780 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.618094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.618903 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.619821 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.620111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.630963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:54.632460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:54.656547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:54.657989 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:54.659268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:54.659268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:54.659268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:54.659268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:54.659268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 624/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:54.663780 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:54.755710 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.775179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0275d-9861-4fbc-8fe1-226d30ce9f84 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.809488 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.810159 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.810446 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.810726 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.811462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:54.811773 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:54.813179 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:54.825871 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 642/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:54.831634 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:54.947142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87cd7-75cc-4605-82c0-0b31cf8e036f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.968423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.984171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.985009 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.985297 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.985575 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.993952 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:54.996171 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:55.011066 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:55.012572 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:55.013762 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 625/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:55.018635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.121939 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72882e97-7bfc-4393-b954-a51a331961d6 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.156946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.157623 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.157904 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.158183 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] PATH_INFO: `/v3/users` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.159018 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:55.159367 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.160909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.171334 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:55.180713 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 643/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:55.186429 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.294927 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576280f-27b3-4115-8a5b-d5abc8210d95 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.333569 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.334506 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.334804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.335058 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.348292 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.349890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:55.370241 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:55.371541 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:55.373390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:55.373390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:55.373390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:55.373390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:55.373390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 626/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:55.378100 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.481517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8161-af5f-4e44-a8ed-154c2f4f7726 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.515886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.516658 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.516951 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.517232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.517963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:55.518276 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.519651 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.532242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 644/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:55.537992 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.570753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:55.644130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00ab-4fd6-4bb9-a771-cd1d27fda18a None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.679898 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.680794 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.681084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.681363 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.689783 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.691531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:55.706929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:55.708280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:28:55.709944 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 627/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:55.714824 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.783734 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:55.828624 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a0b3d-dfef-4332-92ff-f5e50aa2351c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.871992 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.872704 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.872994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.873278 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.874081 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:55.874394 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.875846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fff619-cb7b-4362-9e75-798f371a1bc6 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.889878 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 645/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:55.896114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.989226 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.000378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2847ef-9c00-45fa-af2f-9cbc666aee98 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.036007 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.036929 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] REQUEST_METHOD: `PUT` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.037216 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.037518 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] PATH_INFO: `/v3/projects/9a61be0eb336445a9c847df10323622e/users/850ca515e47a4d0195bba527a2a1696f/roles/fa909fb7c62149c2ba01c1c0ede176c5` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.039481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] RBAC: Authorizing `identity:create_grant(project_id=9a61be0eb336445a9c847df10323622e, user_id=850ca515e47a4d0195bba527a2a1696f, role_id=fa909fb7c62149c2ba01c1c0ede176c5)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:56.040963 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d964-f65c-4221-a857-1c5a3fccf41c None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:56.054986 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 628/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu May 19 20:28:55 2022] PUT /identity/v3/projects/9a61be0eb336445a9c847df10323622e/users/850ca515e47a4d0195bba527a2a1696f/roles/fa909fb7c62149c2ba01c1c0ede176c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:58.371911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec059b-0641-4d99-a7fd-e494b41cdac9 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.372593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec059b-0641-4d99-a7fd-e494b41cdac9 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.372593 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec059b-0641-4d99-a7fd-e494b41cdac9 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.373901 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 646/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:58.379094 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.379508 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.379655 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.397298 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:58.398535 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:58.468970 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 629/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:28:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:28:58.492969 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec059b-0641-4d99-a7fd-e494b41cdac9 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.612049 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec059b-0641-4d99-a7fd-e494b41cdac9 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.630242 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.648469 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.649277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.649565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.649848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.662720 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:58.670868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:58.672145 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:58.673272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 647/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:58.678149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.762632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.783628 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41205b33-9ddf-4656-9051-58b9bcdaf49a None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.819798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.820513 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.820828 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.821109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] PATH_INFO: `/v3/roles` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.821882 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] RBAC: Adding query filter params (name=admin) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.822782 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.824182 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.833108 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 630/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.839136 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.946135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9deb9a4-a518-445c-ba98-b0d58e62ea3c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.980792 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.981610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.981915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.982196 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.993405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.995517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:59.013029 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.014309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.015618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:59.015618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:59.015618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.015618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:59.015618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 648/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:59.020177 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.143551 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e145e57-036f-4269-93d2-ebc866a7c962 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.178356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.179019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.179305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.179591 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] PATH_INFO: `/v3/domains` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.180378 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.180732 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.182540 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.195890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 631/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.202618 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.234088 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.315487 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791d66a-976b-4c58-862e-0dbbbbc8abe3 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.355724 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.356552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.356846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.357128 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.365983 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.367554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:59.383789 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.386161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:28:59.387554 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 649/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:59.391988 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.396842 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.503471 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86acc5-cdf1-438c-a8ed-08547b9529b2 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.539673 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.540352 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.540681 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.541356 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] PATH_INFO: `/v3/users` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.542316 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.542638 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.544214 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.563996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 632/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.570195 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.666185 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.677243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602687-0e9f-44d0-a434-3e4151b64cb0 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.719740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.720428 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.720722 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.721001 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.797527 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.798895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.894845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.905465 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 650/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:28:59.911654 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.985682 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.027043 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243fb9f-8e18-4ff2-a27c-a8372d4e007f None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.062919 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.063753 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.064047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.064335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.076689 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.078173 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:00.096608 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:00.097891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:00.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:00.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:00.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:00.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:29:00.099493 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 633/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:00.103718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.206315 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf3a05-dc40-4b20-b229-a5237fa432d7 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.242082 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.242759 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.243051 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.243336 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] PATH_INFO: `/v3/domains` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.244131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] RBAC: Adding query filter params (name=Default) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.245092 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.246517 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.257876 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 651/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.264335 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.373145 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00dd53f-b4c5-4a1c-aa4a-1ac4c1491680 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.390041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.411112 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.411906 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.412192 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.412549 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.419611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.421088 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:00.436039 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application with Lock( May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:00.437547 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: ERROR keystone.server.flask.application May 19 20:29:00.438863 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 634/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:00.443543 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.496036 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.552584 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5d128-764d-438a-b9ec-50ad6b62187c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.587432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.588285 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.588609 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.588890 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] PATH_INFO: `/v3/projects` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.589783 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.590039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.591513 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58133ea4-1b27-46a1-98bb-0019d0584b47 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.604649 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 652/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.610715 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.725693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1604e7-e41f-4709-a43e-ca1b125ee6fd None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.761849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.762511 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.762798 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.763078 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.763923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] RBAC: Adding query filter params (role.id=fa909fb7c62149c2ba01c1c0ede176c5, scope.project.id=9a61be0eb336445a9c847df10323622e, user.id=850ca515e47a4d0195bba527a2a1696f) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.764240 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.765709 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0683e5b-418e-4f19-9502-bd55fa6e47fe None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.771548 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 635/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/role_assignments?user.id=850ca515e47a4d0195bba527a2a1696f&scope.project.id=9a61be0eb336445a9c847df10323622e&role.id=fa909fb7c62149c2ba01c1c0ede176c5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:03.073828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e11ca-80fc-46c9-ad2d-e81961b74969 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.074406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e11ca-80fc-46c9-ad2d-e81961b74969 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.074406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e11ca-80fc-46c9-ad2d-e81961b74969 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.075797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 653/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:03.081053 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.081580 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.081684 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.099141 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:03.100447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:03.131047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.170851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 636/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:29:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:03.198707 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e11ca-80fc-46c9-ad2d-e81961b74969 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.310758 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e11ca-80fc-46c9-ad2d-e81961b74969 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.322034 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.351256 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.352109 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.352440 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.352730 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] PATH_INFO: `/v3/services/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.353586 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.355177 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:03.368180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:03.370714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:03.372274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 654/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:03.376804 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.490435 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5df02b-5a43-4c41-8ab5-60ae4afa2326 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.526687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.527353 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.527640 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.527918 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.528659 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.528969 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.530340 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed284aa-bfec-4ff8-b28f-7c34e100a5f8 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.535142 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 637/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:03.541309 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.646022 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee3915-d810-4002-b55b-b9ed2a37232c None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.686912 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.687483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.687770 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.688053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] PATH_INFO: `/v3/services` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.688800 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] RBAC: Adding query filter params (type=placement) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.689156 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.690491 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfb628-526e-4974-a482-dca316944a60 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.695372 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 655/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.943399 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f227b8-8174-4cda-99e1-c3b27962f9bf None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.943956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f227b8-8174-4cda-99e1-c3b27962f9bf None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.943956 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f227b8-8174-4cda-99e1-c3b27962f9bf None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.945433 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 638/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:05.951432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc0a2-1023-43af-bddd-35c102a38147 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.951962 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc0a2-1023-43af-bddd-35c102a38147 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.952075 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debc0a2-1023-43af-bddd-35c102a38147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.968711 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-2debc0a2-1023-43af-bddd-35c102a38147 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:05.969895 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-2debc0a2-1023-43af-bddd-35c102a38147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:06.001718 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.040180 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 656/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:29:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:06.063755 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f227b8-8174-4cda-99e1-c3b27962f9bf None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.095047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.169497 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f227b8-8174-4cda-99e1-c3b27962f9bf None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.204454 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.205138 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.205423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.205704 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.206460 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.207864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54938d60-68ed-4c5c-a79a-df5fe8dfed65 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.221588 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 639/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu May 19 20:29:06 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:08.542902 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8348fc-470c-4f6f-a0d4-bbcd14cba55b None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.543476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8348fc-470c-4f6f-a0d4-bbcd14cba55b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.543476 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8348fc-470c-4f6f-a0d4-bbcd14cba55b None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.544931 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 657/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:08.550886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.551432 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.551432 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.568660 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:08.569883 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:08.645978 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 640/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:29:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:08.670095 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8348fc-470c-4f6f-a0d4-bbcd14cba55b None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.786546 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8348fc-470c-4f6f-a0d4-bbcd14cba55b None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.824110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.824870 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.825167 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.825453 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.826122 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.827521 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:08.839375 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:08.840856 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:08.842073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 658/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:08.847041 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.866898 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:08.953652 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d979438-8c1b-408f-8db8-8c0226ccf4a3 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.989766 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.990439 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.990729 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.991019 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.991740 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:08.992051 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.993483 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.999464 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 641/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:09.005564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.039619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.117818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2a343-cacc-46fe-9b96-828a65937c19 None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.153694 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.154383 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.154674 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.154957 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.155768 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] RBAC: Adding query filter params (interface=public, service_id=e857499850e2423a9a8343a5cb418263, region_id=RegionOne) {{(pid=277575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:09.156083 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.157501 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21884335-29e7-4e6b-877f-3679364c177f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.162615 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 659/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/endpoints?service_id=e857499850e2423a9a8343a5cb418263&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:09.170520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.256060 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.276005 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb927-031f-4842-a8d1-d8c72f4e5a50 None admin] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.311217 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.311893 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.312185 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.312504 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.313201 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.314606 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b7961-8b5f-4670-a8e7-d3b980d4c53e None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.321831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 642/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:29:11.588312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803bcc1-d566-47f3-bfcf-a03655980903 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.588917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803bcc1-d566-47f3-bfcf-a03655980903 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.588917 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803bcc1-d566-47f3-bfcf-a03655980903 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.590343 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 660/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:11.595738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.596149 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.596312 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.614238 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:11.615447 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:11.691084 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 643/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:29:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:11.714941 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803bcc1-d566-47f3-bfcf-a03655980903 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.813614 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.821166 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803bcc1-d566-47f3-bfcf-a03655980903 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.860053 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.860911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.861207 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.861492 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] PATH_INFO: `/v3/services/placement` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.862169 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.863552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:11.875738 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:11.877316 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application with Lock( May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: ERROR keystone.server.flask.application May 19 20:29:11.878784 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 661/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:11.883123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.965727 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.989032 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f3a92-2752-485d-87a1-6c8f4edafb00 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.024627 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.025945 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.026229 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.026509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] PATH_INFO: `/v3/services` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.027222 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=277574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.027534 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.028947 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc9e4f-2c04-4f79-8088-03254b3908d0 None admin] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.034100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 644/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.040132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.142470 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a4288-38f5-44d9-8481-2bc09ddaafda None admin] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.177990 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a86a18b340304607b0aafdfa7ad74e79', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.178650 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.178936 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.179211 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.179964 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.181401 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5872-3fc9-434e-923e-5f08a7ba6e9f None admin] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.200312 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 662/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu May 19 20:29:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:22.031552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=30 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.194797 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:21.355130 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4444-1c06-4856-8d43-45fb86c99ad9 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:21.355891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4444-1c06-4856-8d43-45fb86c99ad9 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:21.355891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d4444-1c06-4856-8d43-45fb86c99ad9 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:21.357253 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 645/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:21.363619 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30181-2724-45a8-9130-c8f37257d29c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:21.364187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30181-2724-45a8-9130-c8f37257d29c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:21.364187 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30181-2724-45a8-9130-c8f37257d29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:21.553405 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-b2d30181-2724-45a8-9130-c8f37257d29c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:21.554610 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-b2d30181-2724-45a8-9130-c8f37257d29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:21.813175 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 663/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:30:21.876751 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:21.877479 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:21.878001 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:21.880128 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 646/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 20:30:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:21.891516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab0c8-94b7-4864-8e39-439a65da480c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:21.891829 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab0c8-94b7-4864-8e39-439a65da480c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:21.892110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cab0c8-94b7-4864-8e39-439a65da480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:21.989141 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-c8cab0c8-94b7-4864-8e39-439a65da480c None None] MFA Rules not processed for user `fac286d2b5514467be990cbea641cb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:21.990639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-c8cab0c8-94b7-4864-8e39-439a65da480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:22.014607 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:22.113746 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 664/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 20:30:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:22.124154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:22.126228 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:22.370227 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb854fa-81db-4239-849d-9b40cf6cb2de None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:22.516980 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:22.517713 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:22.518008 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:22.518293 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:22.520387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:22.593523 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:22.670787 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:22.672279 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ebfe-4559-4825-98f9-6a324a419e16 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:22.695424 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 647/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:25.450675 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6f214-a3be-4ab4-80ea-70e413c2fdf8 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:25.451272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6f214-a3be-4ab4-80ea-70e413c2fdf8 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:25.451272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6f214-a3be-4ab4-80ea-70e413c2fdf8 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:25.452885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 665/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:25.458639 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343ef6-70d6-42ca-a382-c9684659d18f None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:25.459159 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343ef6-70d6-42ca-a382-c9684659d18f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:25.459262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343ef6-70d6-42ca-a382-c9684659d18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:25.562851 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-70343ef6-70d6-42ca-a382-c9684659d18f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:25.564018 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-70343ef6-70d6-42ca-a382-c9684659d18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:25.700047 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 648/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:25.756556 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:25.756828 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:25.757110 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:25.758981 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 666/1314] 10.222.0.13 () {54 vars in 964 bytes} [Thu May 19 20:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:25.765694 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8cfd-aaf2-40a6-98f3-f98f28e2de19 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:25.766057 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8cfd-aaf2-40a6-98f3-f98f28e2de19 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:25.766347 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc8cfd-aaf2-40a6-98f3-f98f28e2de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:25.782183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:25.875475 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1bdc8cfd-aaf2-40a6-98f3-f98f28e2de19 None None] MFA Rules not processed for user `fac286d2b5514467be990cbea641cb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:25.876771 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1bdc8cfd-aaf2-40a6-98f3-f98f28e2de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:25.976411 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 649/1315] 10.222.0.13 () {58 vars in 1061 bytes} [Thu May 19 20:30:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:25.985751 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:25.987680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:26.136274 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25b9b1-473d-490a-ada3-66e33febbac7 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:26.250919 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:26.254639 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:26.255456 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:26.255847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:26.256558 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:26.260910 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:26.416644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:26.418146 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a60285-e528-4773-9363-1c3bf02e6fa2 service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:26.442644 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 667/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:29.348103 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.348720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.348720 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.350154 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 650/1317] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:29.356189 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3c1a-de6d-428c-9953-bd175807d54b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.356648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3c1a-de6d-428c-9953-bd175807d54b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.356795 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f3c1a-de6d-428c-9953-bd175807d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.433447 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-fc1f3c1a-de6d-428c-9953-bd175807d54b None None] MFA Rules not processed for user `50716494befb4be495382d33e183829c`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:29.434691 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-fc1f3c1a-de6d-428c-9953-bd175807d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:29.489946 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:29.538191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 668/1318] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:29.582717 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:29.584366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:29.704741 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fed11-e7b8-4878-b6a1-78e2d4a57918 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:29.707716 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:29.793213 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:29.793881 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.794168 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.794445 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.796174 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:29.938237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:29.940651 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482cd7d-cb15-4fc5-b86a-f6a0da3d2d31 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:29.962700 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 651/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:30:32.870714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.871272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.871272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.872742 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 669/1320] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:32.878389 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602bec8-a77a-4119-bb3d-3e860dfc8e41 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.878818 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602bec8-a77a-4119-bb3d-3e860dfc8e41 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.878959 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602bec8-a77a-4119-bb3d-3e860dfc8e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.921841 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:32.949875 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-3602bec8-a77a-4119-bb3d-3e860dfc8e41 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:32.951109 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-3602bec8-a77a-4119-bb3d-3e860dfc8e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:33.055520 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 652/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:33.094933 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:33.096569 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:33.210885 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9494f1-ea6f-4321-8964-9bbda6acbd24 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:33.291854 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:33.292549 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:33.292845 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:33.293131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:33.294846 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:33.303771 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:33.411645 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:33.413135 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbef31-acce-4048-9d97-06b9e83742f9 service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:33.432516 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 670/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:36.656777 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:36.657387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:36.657387 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:36.658810 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 653/1323] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:36.665847 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb327e0-3508-49ed-8b76-ac13a191a384 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:36.666483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb327e0-3508-49ed-8b76-ac13a191a384 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:36.666483 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb327e0-3508-49ed-8b76-ac13a191a384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:36.738490 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-cdb327e0-3508-49ed-8b76-ac13a191a384 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:36.739740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-cdb327e0-3508-49ed-8b76-ac13a191a384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:36.801489 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:36.846235 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 671/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:36.882111 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:36.884042 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:36.996075 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce67654-fd42-4114-ba64-b53f1e87a4b3 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:37.009146 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:37.078114 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:37.078773 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:37.079064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:37.079351 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:37.081064 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:37.212891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:37.214367 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04305cfd-5822-4fb3-90f8-d226c767f274 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:37.244366 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 654/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:39.576331 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.576911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.576911 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.578460 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 672/1326] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:39.583791 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317aa20-fd3e-4781-bd6a-428baa0e9a4f None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.584209 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317aa20-fd3e-4781-bd6a-428baa0e9a4f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.584372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317aa20-fd3e-4781-bd6a-428baa0e9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.635459 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:39.657408 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-b317aa20-fd3e-4781-bd6a-428baa0e9a4f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:39.658688 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-b317aa20-fd3e-4781-bd6a-428baa0e9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:39.764582 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 655/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:39.801232 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:39.802805 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:39.922161 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2eb39b-3fcb-4592-9e6e-46a1397fe230 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:40.000994 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:40.001648 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:40.001934 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:40.002210 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:40.003871 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:40.022857 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:40.139243 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:40.141209 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad3618-bf0c-49bd-9041-85ad8299822d service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:40.163098 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 673/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:42.601658 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.602277 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.603632 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 656/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:42.608892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626131f-bbcc-45cc-ade4-b7af83fbbd4b None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.609367 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626131f-bbcc-45cc-ade4-b7af83fbbd4b None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.609480 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626131f-bbcc-45cc-ade4-b7af83fbbd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.678478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a626131f-bbcc-45cc-ade4-b7af83fbbd4b None None] MFA Rules not processed for user `50716494befb4be495382d33e183829c`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:42.680434 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a626131f-bbcc-45cc-ade4-b7af83fbbd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:42.748318 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:42.780848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 674/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:42.817738 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:42.819317 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:42.931480 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f14e5-d66a-432f-9a13-30f81ccf9415 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:42.954234 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:43.011561 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:43.012212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:43.012531 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:43.012811 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:43.014477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:43.130254 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:43.131725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e2c27-bb8e-4216-8a18-f39cadb0dd66 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:43.159886 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 657/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:45.720270 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:45.720945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:45.720945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:45.722275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 675/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:45.729237 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49191daf-7291-4d44-8856-1ca97554f6c4 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:45.729732 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49191daf-7291-4d44-8856-1ca97554f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:45.729852 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49191daf-7291-4d44-8856-1ca97554f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:45.804702 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:45.808034 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-49191daf-7291-4d44-8856-1ca97554f6c4 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:45.809954 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-49191daf-7291-4d44-8856-1ca97554f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:45.912822 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 658/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:45.949599 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:45.951157 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:46.068272 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ed87b-1fd9-4b10-bbc3-36749c39d56a None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:46.144760 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:46.145406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:46.145692 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:46.145972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:46.147626 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:46.178038 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:46.262190 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:46.263635 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72518390-3df0-463e-9e29-fd9014e3e0c8 service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:46.283811 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 676/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:48.634825 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:48.635423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:48.635423 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:48.636868 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 659/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:48.642204 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434096-7e7a-427f-8c3f-e40a09b28792 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:48.642659 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434096-7e7a-427f-8c3f-e40a09b28792 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:48.642868 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e434096-7e7a-427f-8c3f-e40a09b28792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:48.714825 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-2e434096-7e7a-427f-8c3f-e40a09b28792 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:48.716132 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-2e434096-7e7a-427f-8c3f-e40a09b28792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:48.795105 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:48.818191 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 677/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:48.859710 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:48.861310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:48.972973 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4984d8-61ee-497e-be8c-33a9e6b0837f None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:49.003850 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:49.052333 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:49.053036 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:49.053328 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:49.053612 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:49.056086 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:49.171434 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:49.172938 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13df13b-c6e6-40aa-b5a6-86dde21f4e38 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:49.192670 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 660/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:51.895002 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:51.895672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:51.895672 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:51.897138 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 678/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:51.903071 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8bbd5-3a6a-4a65-9c44-d94dcdd061b3 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:51.903653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8bbd5-3a6a-4a65-9c44-d94dcdd061b3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:51.903653 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8bbd5-3a6a-4a65-9c44-d94dcdd061b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:51.971746 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-cef8bbd5-3a6a-4a65-9c44-d94dcdd061b3 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:51.973022 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-cef8bbd5-3a6a-4a65-9c44-d94dcdd061b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:51.981642 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:52.089503 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 661/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:30:52.125899 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:52.129292 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:52.250886 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184296bc-77c9-4868-b2a9-60f85f0ddda4 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:52.254617 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:52.255275 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:52.255564 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:52.255848 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:52.257673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:52.375611 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:52.377031 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572e9aa-8438-4186-a91a-1fdc99a89a76 service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:52.387945 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=100 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:52.397740 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 679/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:56.822864 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394970-b79c-4992-ad96-0010854b4e8c None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:56.823449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394970-b79c-4992-ad96-0010854b4e8c None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:56.823449 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d394970-b79c-4992-ad96-0010854b4e8c None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:56.824909 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 662/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:56.830378 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5a9e-4f89-41e2-aa30-3038ff30222c None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:56.830818 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5a9e-4f89-41e2-aa30-3038ff30222c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:56.830923 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5a9e-4f89-41e2-aa30-3038ff30222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:56.881360 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-a72c5a9e-4f89-41e2-aa30-3038ff30222c None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:56.882565 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-a72c5a9e-4f89-41e2-aa30-3038ff30222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:56.964484 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 680/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:30:57.002923 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d394970-b79c-4992-ad96-0010854b4e8c None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:57.014677 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d394970-b79c-4992-ad96-0010854b4e8c None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.018244 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.018887 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.019215 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.019448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.030126 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:57.031570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a39f53-4ca2-4b16-b97f-a7cd27f4e3b9 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:57.042198 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 663/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:30:59.939678 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190736b-341f-47b5-a6f5-99dbf4dea81d None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:59.940255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190736b-341f-47b5-a6f5-99dbf4dea81d None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:59.940255 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8190736b-341f-47b5-a6f5-99dbf4dea81d None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:59.941681 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 681/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:59.947838 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8c82-5586-4231-923b-b83bf5697856 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:59.948310 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8c82-5586-4231-923b-b83bf5697856 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:59.948448 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e8c82-5586-4231-923b-b83bf5697856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:59.978552 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-fe7e8c82-5586-4231-923b-b83bf5697856 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:59.979831 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-fe7e8c82-5586-4231-923b-b83bf5697856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:59.987972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 664/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:30:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:00.025538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190736b-341f-47b5-a6f5-99dbf4dea81d None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.037590 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8190736b-341f-47b5-a6f5-99dbf4dea81d None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.043089 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.043690 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.043904 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.044194 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.054459 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:00.055963 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299612aa-b5fb-4f5d-8384-57abd70c1bf4 service neutron] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:00.066127 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 682/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:31:04.584740 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3452f9-91dc-465f-8343-6ae1f0bd55cd None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:04.585268 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3452f9-91dc-465f-8343-6ae1f0bd55cd None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:04.585359 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3452f9-91dc-465f-8343-6ae1f0bd55cd None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:04.587305 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 665/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:04.594071 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d867-84b3-4bdb-975f-dd1a41470b2f None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:04.594598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d867-84b3-4bdb-975f-dd1a41470b2f None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:04.594729 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4d867-84b3-4bdb-975f-dd1a41470b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:04.614342 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-82c4d867-84b3-4bdb-975f-dd1a41470b2f None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:04.615575 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-82c4d867-84b3-4bdb-975f-dd1a41470b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:04.624039 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 683/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:04.663089 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3452f9-91dc-465f-8343-6ae1f0bd55cd None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:04.674890 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3452f9-91dc-465f-8343-6ae1f0bd55cd None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:04.678477 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fac286d2b5514467be990cbea641cb5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:04.679123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:04.679409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:04.679687 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:04.689076 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:04.690558 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15927c3-3f88-4c0c-8368-fe425593e376 service neutron] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:04.701101 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 666/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:31:10.502920 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99219008-01df-4f06-9ddf-a53e1a21f788 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:10.504323 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99219008-01df-4f06-9ddf-a53e1a21f788 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:10.517892 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99219008-01df-4f06-9ddf-a53e1a21f788 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:10.520714 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 684/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:31:10 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:10.535763 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2775b4-e48e-4eef-8625-c5d38c05fc28 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:10.536725 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2775b4-e48e-4eef-8625-c5d38c05fc28 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:10.537212 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2775b4-e48e-4eef-8625-c5d38c05fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:10.659711 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-7c2775b4-e48e-4eef-8625-c5d38c05fc28 None None] MFA Rules not processed for user `850ca515e47a4d0195bba527a2a1696f`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:10.660993 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-7c2775b4-e48e-4eef-8625-c5d38c05fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:10.719370 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=100 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:10.763183 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 667/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:14.609406 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=56 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:15.503263 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d57f6-251f-467b-b6c6-3bc6e3b8fdc0 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:15.503974 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d57f6-251f-467b-b6c6-3bc6e3b8fdc0 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:15.504551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d57f6-251f-467b-b6c6-3bc6e3b8fdc0 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:15.506514 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 685/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:15.512544 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096abc-7bfc-4ce5-84e1-e82b8c94be58 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:15.513065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096abc-7bfc-4ce5-84e1-e82b8c94be58 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:15.513155 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be096abc-7bfc-4ce5-84e1-e82b8c94be58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:15.534693 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-be096abc-7bfc-4ce5-84e1-e82b8c94be58 None None] MFA Rules not processed for user `850ca515e47a4d0195bba527a2a1696f`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:15.536372 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-be096abc-7bfc-4ce5-84e1-e82b8c94be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:15.547082 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 668/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:31:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:17.122152 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de697-a217-4bed-a22f-ee42248632c5 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:17.122673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de697-a217-4bed-a22f-ee42248632c5 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:17.122673 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de697-a217-4bed-a22f-ee42248632c5 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:17.124302 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 686/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu May 19 20:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:17.129553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a898-3e50-493d-9165-88edccccbd0a None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:17.129996 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a898-3e50-493d-9165-88edccccbd0a None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:17.130231 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458a898-3e50-493d-9165-88edccccbd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:17.149570 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-b458a898-3e50-493d-9165-88edccccbd0a None None] MFA Rules not processed for user `850ca515e47a4d0195bba527a2a1696f`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:17.151027 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-b458a898-3e50-493d-9165-88edccccbd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:17.160026 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 669/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu May 19 20:31:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:31:21.073894 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe6f7-37d5-417f-a8bb-5aaa4c504955 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:21.074452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe6f7-37d5-417f-a8bb-5aaa4c504955 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:21.074452 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcfe6f7-37d5-417f-a8bb-5aaa4c504955 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:21.075839 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 687/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:21.081721 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c32-e5b2-4460-863c-1e04af266e18 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:21.082280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c32-e5b2-4460-863c-1e04af266e18 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:21.082280 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f91c32-e5b2-4460-863c-1e04af266e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:21.107972 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-a8f91c32-e5b2-4460-863c-1e04af266e18 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:21.109298 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-a8f91c32-e5b2-4460-863c-1e04af266e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:21.123065 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 670/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:21.191552 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87986-7d72-4706-adae-6231ee470c05 None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:21.191844 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87986-7d72-4706-adae-6231ee470c05 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:21.192131 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87986-7d72-4706-adae-6231ee470c05 None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:21.194074 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 688/1358] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:21.220846 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29342c00-109e-41fe-a1c7-79667fece588 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:21.221134 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29342c00-109e-41fe-a1c7-79667fece588 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:21.221754 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29342c00-109e-41fe-a1c7-79667fece588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:21.279562 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-29342c00-109e-41fe-a1c7-79667fece588 None None] MFA Rules not processed for user `d4759f19e262476a8c4c4a99b43daa13`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:21.281343 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-29342c00-109e-41fe-a1c7-79667fece588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:21.387251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 671/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:31:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:21.395598 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa87986-7d72-4706-adae-6231ee470c05 None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:21.411958 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa87986-7d72-4706-adae-6231ee470c05 None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:21.418949 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4759f19e262476a8c4c4a99b43daa13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:21.419677 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:21.420046 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:21.420432 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:21.429423 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:21.432047 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682d40f-4e13-434a-adc5-94cd573a3001 service nova] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:21.452431 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 689/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:31:23.989407 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8b87-0e4a-4438-826d-36785b3d4d60 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:23.989849 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8b87-0e4a-4438-826d-36785b3d4d60 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:23.989962 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe8b87-0e4a-4438-826d-36785b3d4d60 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:23.991390 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 672/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:23.997906 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bc2dd-e808-4ff9-a43c-20856bfbf895 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:23.998457 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bc2dd-e808-4ff9-a43c-20856bfbf895 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:23.998693 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bc2dd-e808-4ff9-a43c-20856bfbf895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:24.018984 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-de1bc2dd-e808-4ff9-a43c-20856bfbf895 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:24.020294 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-de1bc2dd-e808-4ff9-a43c-20856bfbf895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:24.032588 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 690/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:24.075121 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe8b87-0e4a-4438-826d-36785b3d4d60 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:24.090521 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe8b87-0e4a-4438-826d-36785b3d4d60 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:24.105123 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4759f19e262476a8c4c4a99b43daa13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:24.105891 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:24.106262 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:24.106553 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:24.115935 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:24.118481 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0816a9-aafa-4c2b-95d7-b95842735ab0 service nova] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:24.128664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 673/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:31:26.773317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbf07-29cf-4571-8feb-c7d442199dce None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:26.774123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbf07-29cf-4571-8feb-c7d442199dce None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:26.774123 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedbf07-29cf-4571-8feb-c7d442199dce None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:26.775380 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 691/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:31:26.781000 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850adef-133c-4ead-856b-3032d5c9e236 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:26.781806 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850adef-133c-4ead-856b-3032d5c9e236 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:26.782308 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850adef-133c-4ead-856b-3032d5c9e236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:26.802324 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-1850adef-133c-4ead-856b-3032d5c9e236 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:26.803845 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-1850adef-133c-4ead-856b-3032d5c9e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:26.813218 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 674/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:26.862441 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959705c-218d-4d9c-a4e4-d5fe8e7b251c None None] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:26.863035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959705c-218d-4d9c-a4e4-d5fe8e7b251c None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:26.863478 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959705c-218d-4d9c-a4e4-d5fe8e7b251c None None] PATH_INFO: `/` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:26.865462 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 692/1366] 10.222.0.13 () {54 vars in 961 bytes} [Thu May 19 20:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:26.871966 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b47fd-4d41-4e38-ad08-3b9323fadc39 None None] REQUEST_METHOD: `POST` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:26.872266 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b47fd-4d41-4e38-ad08-3b9323fadc39 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:26.872598 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b47fd-4d41-4e38-ad08-3b9323fadc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:26.896577 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.auth.core [None req-549b47fd-4d41-4e38-ad08-3b9323fadc39 None None] MFA Rules not processed for user `d4759f19e262476a8c4c4a99b43daa13`. Rule list: `[]` (Enabled: `True`). {{(pid=277574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:26.897914 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.fernet_utils [None req-549b47fd-4d41-4e38-ad08-3b9323fadc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:26.917855 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 675/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Thu May 19 20:31:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:26.925551 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959705c-218d-4d9c-a4e4-d5fe8e7b251c None None] Authenticating user token {{(pid=277575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:26.941799 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959705c-218d-4d9c-a4e4-d5fe8e7b251c None None] Validating token access rules against request {{(pid=277575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:26.948663 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4759f19e262476a8c4c4a99b43daa13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:26.949408 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] REQUEST_METHOD: `GET` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:26.949779 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:26.950148 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:26.959589 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:26.961545 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e951c-57a9-42f7-9de4-1ca1a37856e9 service nova] RBAC: Authorization granted {{(pid=277575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:26.973277 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 693/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:31:28.206865 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fea83-5c0f-4ba8-b4cb-f0dbca55ae50 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:28.207509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fea83-5c0f-4ba8-b4cb-f0dbca55ae50 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:28.207509 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fea83-5c0f-4ba8-b4cb-f0dbca55ae50 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:28.208974 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 676/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:31:28.215486 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8b8c-bce6-4bd4-9128-ab2029b10de0 None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:28.216035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8b8c-bce6-4bd4-9128-ab2029b10de0 None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:28.216035 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e8b8c-bce6-4bd4-9128-ab2029b10de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:28.237317 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-0c0e8b8c-bce6-4bd4-9128-ab2029b10de0 None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:28.238550 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-0c0e8b8c-bce6-4bd4-9128-ab2029b10de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:28.248073 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 694/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:31:29.347122 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5ab1-00a5-42b0-9a92-0be077ac89b3 None None] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:29.347722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5ab1-00a5-42b0-9a92-0be077ac89b3 None None] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:29.347722 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5ab1-00a5-42b0-9a92-0be077ac89b3 None None] PATH_INFO: `/` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:29.349100 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 677/1371] 10.222.0.13 () {54 vars in 926 bytes} [Thu May 19 20:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:31:29.354972 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fd605-e1c3-4583-b2ac-8854698e1dac None None] REQUEST_METHOD: `POST` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:29.355421 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fd605-e1c3-4583-b2ac-8854698e1dac None None] SCRIPT_NAME: `/identity` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:29.355538 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fd605-e1c3-4583-b2ac-8854698e1dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=277575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:29.378680 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.auth.core [None req-d43fd605-e1c3-4583-b2ac-8854698e1dac None None] MFA Rules not processed for user `a86a18b340304607b0aafdfa7ad74e79`. Rule list: `[]` (Enabled: `True`). {{(pid=277575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:31:29.379909 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG keystone.common.fernet_utils [None req-d43fd605-e1c3-4583-b2ac-8854698e1dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=277575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:31:29.388915 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: [pid: 277575|app: 0|req: 695/1372] 10.222.0.13 () {58 vars in 1023 bytes} [Thu May 19 20:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:31:29.441645 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5ab1-00a5-42b0-9a92-0be077ac89b3 None None] Authenticating user token {{(pid=277574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:29.457409 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5ab1-00a5-42b0-9a92-0be077ac89b3 None None] Validating token access rules against request {{(pid=277574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:29.463358 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4759f19e262476a8c4c4a99b43daa13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a61be0eb336445a9c847df10323622e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=277574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:29.464023 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] REQUEST_METHOD: `GET` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:29.464320 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] SCRIPT_NAME: `/identity` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:29.464646 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=277574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:29.474594 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:29.476251 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85474fa2-d37c-4747-804b-9e640a166b59 service nova] RBAC: Authorization granted {{(pid=277574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:29.486664 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: [pid: 277574|app: 0|req: 678/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Thu May 19 20:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:31:39.373019 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: DEBUG dbcounter [-] [277575] Writing DB stats keystone:SELECT=21 {{(pid=277575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:39.482211 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: DEBUG dbcounter [-] [277574] Writing DB stats keystone:SELECT=86 {{(pid=277574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:56.414965 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: Thu May 19 20:31:56 2022 - graceful shutdown triggered... May 19 20:31:56.415611 dv-cinder-840678-11-iscsi devstack@keystone.service[277574]: Gracefully killing worker 1 (pid: 277574)... May 19 20:31:56.416081 dv-cinder-840678-11-iscsi devstack@keystone.service[277575]: Gracefully killing worker 2 (pid: 277575)... May 19 20:31:56.418175 dv-cinder-840678-11-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:31:57.416047 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: worker 1 buried after 1 seconds May 19 20:31:57.416047 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: worker 2 buried after 1 seconds May 19 20:31:57.416047 dv-cinder-840678-11-iscsi devstack@keystone.service[277564]: goodbye to uWSGI. May 19 20:31:57.417696 dv-cinder-840678-11-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:31:57.418241 dv-cinder-840678-11-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.